← Journey

Checking Is Cheaper Than Generating

The research finding that makes the whole GVS architecture work: verifying a claim against evidence is fundamentally easier than producing a correct one from scratch.

There’s a 2021 paper from Cobbe et al. that has a striking result: a 6B model with a verifier (best-of-100 sampling, picking the answer the verifier scores highest) matches a 175B model on grade-school math. That’s a 29x parameter gap, closed by running the small model multiple times and checking the answers. Recognition is cheaper than production.

The Research Pattern

The same result shows up in different forms across the literature.

GenRM from DeepMind: GSM8K accuracy goes from 73% to 93.4% using a generative reward model to select among candidates. Snell 2024: a small model with compute-optimal test-time search beats a 14x larger model on MATH-500. ThinkPRM-1.5B trained on 8,000 examples beats discriminative process reward models trained on 100 times more data.

The pattern is consistent enough that it’s not a cherry-picked result. You can spend the asymmetry by running more searches and checking them, or you can spend it by using a much smaller model and being careful about which answers you emit.

Three Required Conditions

This only works if three conditions hold.

First, the task has to be verifiable. There needs to be a compact, checkable signal for correctness. Math, code with tests, and factual claims against retrieved evidence all qualify. Open-ended creative writing doesn’t. Opinion questions don’t. There’s no small-model verifier that reliably judges creative quality, and the research reflects that: human-LLM inter-annotator agreement on creative writing is around 0.43, which means even humans don’t agree with each other well enough for a verifier to be trained on their labels.

Second, the verification has to be external. This one is counterintuitive. You might expect that if a model generates a wrong answer, asking it “are you sure?” would help. It doesn’t. Huang et al. showed that same-model intrinsic self-correction fails systematically. A more recent paper showed the framing matters in a striking way: external framing (“does this CLAIM follow from this EVIDENCE?”) outperforms internal framing (“did you make an error in your reasoning?”) by 77 percentage points. The model is evaluating a claim against external evidence, not its own reasoning. That’s a different cognitive operation, and small models can do it reliably.

Third, the verification signal should be grounded or rule-based. Neural reward models get gamed. Under RL, exploit rates go from 0.6% to 13.9% over training runs. The Reward-Bench paper showed that the best available reward models score 46.6% on hard style-sensitive benchmarks, which is below random. DeepSeek-R1 uses rule-based rewards for this reason: exact-match on math answers, unit test pass rates on code, no neural reward model in the loop.

Our Verifier at 1.7B

For our surface (grounded factual question answering), the right verifier is an entailment check: does the candidate answer follow from the retrieved evidence? This is a well-defined binary question.

Small models are good at it. MiniCheck-FT5 at 770M parameters does it within 0.6% of GPT-4 at roughly 446 times lower cost. AlignScore at 355M beats GPT-4-based G-EVAL on factual consistency.

We measured this on our own 1.7B model specifically rather than trusting the published numbers on other models. On easy negatives (claims that clearly aren’t supported by the evidence), the model was right 100% of the time (10/10). On hard negatives (subtly off, plausible but unsupported, numeric boundary cases), it was right 90% of the time, with one miss on a decimal bound edge case.

That 90-100% accuracy at 1.7B is sufficient to make the GVS loop work. The model is an unreliable generator at 1.7B. But it’s a reliable verifier for grounded claims. The architecture spends the model’s strength where it has strength.

CPU Latency Forces the Design

The interaction between verification and CPU latency forces the specific design we use.

Best-of-128 sampling and checking all 128 candidates would give higher answer quality. But on a CPU box, generating 128 answers takes around 134 seconds. That’s not interactive. So the loop is: generate one candidate, verify it. If it passes, emit. If it fails, generate a few more (2-3), verify each, keep the best. If none pass, abstain. The search path is rare in practice. Most grounded questions get answered on the first try.

Abstention is a feature. A system that says “I don’t have reliable information about that” is more trustworthy than one that always produces an answer. Frontier models still hallucinate citations between 14% and 95% of the time depending on the domain and how you measure it. A system that emits only verified claims and abstains otherwise is competitive on trustworthiness even if it’s much smaller.

Key takeaways:

Next: the benchmark that tests all of this on real data.

← All entries