How a figure earns its grade

Every number in an answer carries a provenance grade, computed by the server, from evidence, before you ever see it. This page explains what each grade proves, what it deliberately does not, and why a perfect score is rare and should be.

Most systems that put an AI answer in front of you give you one signal: a checkmark, or nothing. A checkmark that means "we ran our process" and a checkmark that means "we recomputed every number" look identical, and the first one is much easier to earn. Plumbline separates them and shows you both.

The line you are reading

Checks passed 2 of 3 figures re-checked against SQL · weakest grade: derived

A real line, from an answer naming the highest and lowest paid employee and the gap between them. Two figures were read from the store. The third, the gap, was computed by the server from those two, so it carries its arithmetic instead of a direct match.

Verified 12 of 12 figures re-checked against SQL

Also real, from an answer about role pay bands. The strong case, and the only case that earns the word: every figure in the answer was recomputed from the database and agreed.

The headline word is a claim about process. The line beside it is a claim about arithmetic. They are different claims and they are allowed to disagree, which is exactly why they are shown separately.

Checks passed

Every stage of the pipeline ran. The answerability gate allowed it, the refusal guard did not fire, and no figure contradicted the store. The answer was permitted to publish.

Verified

All of the above, and every quantitative figure in the text was independently recomputed from the database and matched. Printed only when coverage is complete.

The panel steps the word down to "Checks passed" the moment coverage is short, and moves the count to the front of the line, because the count is the reason the word got weaker. Burying it behind reassuring language is how a partial result comes to read as a total one.

The count: "2 of 3"

3 is every quantitative figure in the published answer text: dollar amounts, headcounts, medians, percentages. It is a count of numbers in the prose, not of rows retrieved or facts consulted. A longer answer has a bigger denominator, so coverage fractions are not comparable across answers of different length: the second specimen above says 12 of 12 because that answer used twelve figures, not because it is twelve times better.

2 is how many of those cleared the highest bar. That bar is deliberately narrow:

  1. Before the model writes a word, the server runs a saved SQL query and publishes the result as a named fact key with an exact value, for example role_median_income_cents.staff_engineer = 18000000.
  2. The model is handed that list of claimable keys and writes its answer.
  3. The server binds a figure in the finished text to one of those keys, so it knows which concept that number is asserting.
  4. The server re-derives the value from the store and compares. Equal, so the figure is graded verified.
The point of step 4

This is not "the model was given the right number, so we trust it used the right number." It is "we computed the number a second time, independently, and got the same answer." A figure that skips step 4 has not been checked, however plausible it looks.

The five grades

Grades are ordered. The headline reports the weakest one in the answer, never the average.

GradeRungWhat it proves
verified 5 Recomputed from a SQL seed for its concept, and agreed. The strongest statement the server can make about a number.
corroborated 4 Matched verified evidence, or a cross-check re-derived it and agreed, but no primary SQL seed was bound to it.
derived 3 Computed from verified figures by arithmetic the answer shows: a difference, a ratio, a share, a unit conversion, or a total across printed rows. Not read from the store, so not verified, but the receipt names the operands and the operation and you can recompute it.
no verified source 2 Bound to no checkable concept and absent from the supplied facts. This means "we did not verify it", not "we found it wrong", which is why it sits above conflict rather than at zero.
conflict 1 The figure disagrees with what the store says for that concept. Actively wrong, and an absorbing floor.

Taking the minimum rather than the mean is a deliberate choice. An average lets fourteen good numbers bury one wrong one. A minimum cannot: a single conflicting figure drags the whole headline to the floor no matter what surrounds it.

A grade that used to be here

An earlier version of this page described a rung called traceable: a figure whose digits appeared in the facts the server composed, without being bound to any particular fact. That rung is gone. It described the checker failing to ask rather than a property of the number, and a block this server generates has no figure without an owner in it. What was filed there is now either verified (it matches a value the server computed) or derived (the server can recompute it), and anything neither has no verified source.

Why full coverage is rare, and why that is honest

A low fraction usually means the bar is high, not that something broke. A figure only becomes re-checkable if a fact seed exists for that exact concept, and a seed must be an exact integer, in whole counts or in cents.

That constraint is what makes a match a proof rather than an approximation. It is also what caps coverage, and one consequence is worth stating plainly:

Structural limit

Percentages can never be seeded. A deviation of 23.4% from a role median is neither a whole count nor a whole number of cents, so no seed can ever hold it, and it cannot be verified the way a count or a cents figure is. What it can be is derived: where the server computed the percentage itself, it files the operands and the operation, and the figure publishes with arithmetic you can redo. Where nobody computed it, the numerator and denominator are still each verifiable on their own; the ratio is not.

So a question whose answer is mostly percentages has a low ceiling by construction. The system reports that ceiling instead of rounding up to a green checkmark, and it does not gate the answer either: the figure is published, with its grade attached, and you decide what it is worth.

Reading the line's silences

The note prunes itself. It prints N derived, N corroborated by cross-check, and N conflicts with the store only when those counts are above zero. So the absence of a clause is information.

Checks passed
2 of 3 figures re-checked against SQL · weakest grade: derived

No "corroborated" clause and no "conflicts" clause, and a weakest grade of derived. Those facts together pin down the whole distribution without opening anything: two figures at rung 5, one the server recomputed at rung 3, nothing anywhere else.

A worked example

The run behind the first specimen. The question asked who the highest paid employee is and how large the gap is to the lowest paid.

Checks passed 2 of 3 figures re-checked against SQL · weakest grade: derived

Answered from the pinned demo snapshot.

Two figures cleared: $518,878 and $31,288, the highest and lowest annualized pay among currently active employees. Both are named fact keys, both were recomputed from the database, and both matched.

The third is the gap, $487,590. No query produced that number; the answer worked it out by subtraction. The server did not take the answer's word for it and did not refuse it either: it recomputed the subtraction from the two figures it had already verified, confirmed the result was the number printed, and published it as derived, with a disclosure saying in plain words that one figure was computed from verified figures rather than read from the store. That is the whole of rung 3: not a weaker kind of evidence, a different kind, and one you can redo yourself.

Four other disclosures rode along, none of them about the arithmetic: 12 people report pay on a non-annualizable basis and sit outside the salary distribution, 12 more are paid with no salary rate on file, every role band in this dataset rests on two people or fewer, and the named extremes cover active employees only, excluding 36 income-bearing people on leave, onboarding, prehire or terminated. The answer is three figures long. The context needed to read it honestly is longer, and it publishes anyway.

What we do not claim


The same grades are visible per figure inside the answer panel: click any box in the provenance graph to see where that specific number came from, in plain language. See connect your LLM for the paste-ready setup, or the support page to ask about a specific run.