Fact-Checking

Jev, a Judge to Fact-Check the Writer

September 23, 2026 4 min read

The posts in my latest batch were written by a large model and checked by a small one that cannot write a sentence.

That is not a slogan, it is the architecture. The large model is Claude Opus 5.5, which costs $4 per million input tokens. The small one is Jev, from a company called TypeSafe, which costs $0.042 per million input tokens, with output tokens free. Roughly 95 times cheaper on input. And the part that does the arithmetic is neither of them. It is code.

Here is how it works, what it caught, and where it still fails.

A model that only answers questions

Jev is what TypeSafe calls a System One model. It does not generate text or explanations. You give it some state, a piece of text or a JSON object, and one or more typed questions. It returns typed answers: the probability that a yes-or-no question is yes, a choice among options you define with a probability for each, or a score on levels you describe.

That sounds limiting until you notice how much of fact-checking has exactly that shape. Is this sentence a factual claim? Does this passage support this claim, contradict it, or say nothing about it? Those are judgments, not writing.

The pipeline

I built the checker on September 22, after doing the same job for my September batch with a workflow of 36 agents running on expensive models. It runs in five steps.

First, triage. Jev reads the draft sentence by sentence and says which ones assert something checkable. Opinions, questions and hypotheticals drop out.

Second, evidence. For each claim, code pulls the three passages from each source that share the most words and numbers with it, and merges them in page order. I learned this the hard way: a sentence carrying five facts spread over two paragraphs can never be confirmed by a single three-sentence window.

Third, judgment. Jev reads the claim against each source's evidence and answers supports, contradicts, or not enough information.

Fourth, conflict rules, written in code. A contradiction from one source beats a confirmation from another; that rule came from a real case in which two official pages gave two different dates for the same decision. And a contradiction only counts if the evidence contains at least half the words of the claim, so the judge is looking at the same fact. On my first cases, true contradictions sat at 0.71 and 0.73, false ones at 0.36 or below. I tried asking Jev "is this the same fact?" instead. It rated a real contradiction at 0.43, so I went back to counting words.

Fifth, and most important: numbers never go to the model. Every figure in a claim must appear in the evidence, or follow from two figures in the evidence by one operation, or by a unit conversion. Tolerance is 1.5 percent, 10 percent when the sentence says "about." A claim whose figures all check out by calculation is settled by the calculation.

What it caught

On a real article from my September batch, already scheduled, it found two errors out of two, with no false alarm. It took six seconds and cost about a third of a cent.

On a correct article, it raised one false alarm: an example I had made up to illustrate a point, which was logically absent from the sources.

On that same article with three errors injected, among them a wrong amount and a wrong number of hours, it caught two of the three.

Three articles is not a calibration. I say so in the documentation and I will say it here. It is a start.

It also did a smaller job this week. Before I wrote anything, I gave Jev the 29 tech headlines of the week collected by my news radar and asked it to rate each one as a subject for a post aimed at developers and founders. Twenty-nine calls, a little over twelve thousand tokens, and a bill below a tenth of a cent. It put the model price war and a lawsuit against OpenAI at the top, which is where I would have put them.

Where it fails

It does not search. It checks a draft against the sources I give it. A claim with no source comes back "not confirmed," not "false."

It does not know what exists. If a draft mentions a software version that was never released, Jev will only flag it if a source says something different. The rule I already had, look up every version number before publishing, still stands.

And it has a known blind spot. A short sentence that carries only a date shares too few words with any passage, so its contradiction gets filtered out by the same-fact rule and it comes back "not confirmed." A person has to read those. Running it on this very batch surfaced a cousin of the same problem: a percentage that was correctly in the source got marked as "verified by calculation" through a coincidental ratio of two unrelated numbers. The figure was right; the proof was not. That one goes on my list.

The principle

The design has one idea underneath it: the expensive model should only see what the cheap one could not settle. Code does the counting. The judge does the reading. The writer does the writing. And the uncertain remainder, which is usually short, goes to a stronger model or to me.

A year ago I would have used one big model for all of it and called the result "reviewed." Now the review costs less than the coffee I drink while reading its report, and it tells me exactly which sentences it could not vouch for.