How Plumbline works

A plain walk through what is built before any question arrives, and what happens when one does. Written from the implementing code.

In short

Plumbline answers questions about one pinned copy of a company's workforce data. A language model drafts each answer, but it is handed its material and has no database access, and every figure in the finished answer is checked afterward against values computed by registered SQL, then published with its grade. Anything the data cannot support is refused rather than guessed.

Vectors route questions and rank context; they do not supply or verify answer figures.

Before any question arrives

Everything the runtime uses is authored and built ahead of time. Nothing in this chapter involves a visitor question; it is the work that makes questions answerable later.

A pinned snapshot of the data

Plumbline starts from a snapshot: one dated copy of a company's workforce data pulled from a payroll or HR provider. Ingest writes the employees and any payroll records into SQL tables and pins them; every later answer names the snapshot it was computed from.

From those same rows the server also writes short prose documents, one per subject it knows how to describe, such as a company overview, a compensation distribution summary, and one document per department change. Each document is embedded into a vector and stored beside the tables. The two are different jobs: the documents give the drafting model its reading material, while the SQL side is the only thing a figure can be verified against. Because both views come from the same rows, the server can also reconcile them against each other. Categories the provider does not expose produce no documents at all, so questions about them are refused rather than guessed.

The SQL fact catalog and its seeds

Separately, operators register every SQL query the server may present as fact provenance in one catalog, keyed by a stable query id. The SQL a trace shows, the SQL the fact tool executes, and the SQL registered for audit are one text.

When a registered query runs, each value it publishes is stored under a stable name. That stored, named value is called a seed. Seeds and recorded result cells are the only things a figure in a finished answer can later be verified against.

Question families: the topics it can recognize

The embedding router's known vocabulary is the authored question-family examples described below. A question that matches none of them confidently is not simply rejected: it takes a fallback path, either the planner's own classification or a cautious best guess, and the answerability gate still decides from real retrieved evidence whether anything can be answered at all.

question family

A named topic the router can recognize, defined by authored example phrasings embedded once at startup; recognizing one selects the work that runs.

Plumbline does not try to understand arbitrary questions. Its operators write down, ahead of time, the families of questions it can handle, each as a list of real example phrasings, such as asking for total headcount or for compensation outliers. At startup the server embeds those examples once; that small labeled set is the embedding router's entire known vocabulary. A new question is embedded the same way and lands in the family whose examples it sits closest to; a question it cannot place confidently takes a fallback path instead, where the planner classifies it or a cautious best guess is used, and the answerability gate still decides whether an answer is possible. Whichever way the family is chosen, everything that follows, which SQL runs and how context is searched, is chosen by that family.

What it never claims

that the question is answerable (the answerability gate decides that from real retrieved evidence)
anything about any figure in the answer

Why classification exists

Classification is not decoration; it is how the right work gets selected. Route a headcount question as a payroll question and the search reads the wrong documents, finds none of its required evidence, and refuses a question the data could answer. The families exist so that the search, the required evidence, and the SQL work all match the question actually asked.

retrieval profile

The authored per-family search recipe: the phrases to search with, the document types that count as required evidence, and guidance for how the answer should read.

Once a family is chosen, its retrieval profile takes over. The profile is authored, not learned: it names the search phrases to anchor on, the document types that must be present for the question to be answerable at all, and how the finished answer should read. A compensation question, for example, requires compensation distribution documents to be found before the gate will allow an answer. The profile shapes the search; it contributes no numbers.

What it never claims

that its search phrases are facts (they only steer the document search)
that any figure is correct

Where the domain knowledge comes from

All of the domain knowledge above is authored in code and reviewed like code: the example phrasings, the search anchors, the required evidence lists, the registered SQL, and the seed names. None of it is learned from visitor questions. The embedding model contributes no knowledge of its own; it only measures how close two pieces of text sit.

The one tuned part is calibrated, not guessed. At calibration time the server embeds a list of deliberately unanswerable probe questions, such as asking for a customer satisfaction score no HR snapshot contains, and measures where the active embedding model places them. The routing thresholds are derived from those measured distributions, so a question about something the corpus cannot answer either takes the cautious fallback path or is refused by the gate, whichever the evidence supports.

When a question arrives

The runtime path is six steps plus one standing disclosure. The panel expander named "How this answer was made" shows the same steps filled in with the live record of the answer you are reading; this chapter tells the general story once.

Step 1: route the question

The question is embedded and compared against the stored family examples. The best match picks the family, and with it the SQL work and the retrieval profile.

question-routing similarity

How closely the question matches a known question family, measured by comparing embeddings; it picks which registered work runs.

Before the server can compute anything, it must decide what kind of question this is. An embedding model turns the question into a vector, and the server compares that vector against stored vectors for the question families it knows. The best match selects the family, and the family selects which registered SQL queries and which document-retrieval profile will run. That is the entire job of this score: choosing the path. No figure and no figure grade ever comes from it.

What it never claims

anything about any figure in the answer
that documents about the question exist (that is document retrieval, a separate search with its own scores)

Step 2: compute SQL facts

The family's registered queries run against the pinned snapshot and publish named values, each carrying its recorded query id.

SQL fact

A value computed by a registered SQL query against the pinned data snapshot; the only kind of source that can make a figure verified.

The routed question family dispatches to exactly one registered metric path, which runs approved catalog queries against the pinned snapshot in Postgres. Each output is a named value with a recorded query id: an SQL fact. These are the load-bearing numbers. When the finished answer is checked, a figure earns the verified grade only by matching one of these stored values or a cell in their recorded result sets.

What it never claims

that the provider's underlying data is itself correct (the fact is exact relative to the snapshot)
that every number in the answer came from a fact like this (each figure is checked separately)

SQL fact seed

The exact named stored value behind one SQL fact; the thing a verified figure matched.

Every SQL fact stores its value under a stable name, the seed. Verification is a plain equality check between a number in the answer and a seed or a recorded result cell. There is no model judgment and no similarity in this step; the digits either match a stored value or they do not.

What it never claims

that a merely equal but untagged number was derived from it

SQL result set

The recorded table a registered query returned; any cell in it can verify a matching figure.

Some registered queries return whole tables, not one value. The server records the table with the answer. A figure in the answer that equals one of the recorded cells is verified the same way a seed match is, because the cell came from the same approved query path.

What it never claims

that rows from generated or raw SQL lanes carry the same weight (those are separately labeled)

Step 3: retrieve context documents

The profile's search runs over the stored prose documents with hybrid vector and text ranking. The winners become the context the drafting model will read.

retrieved context document

A stored prose document selected by hybrid vector and text search to give the model background; context, never figure authority.

Separately from the SQL path, the server searches its document store for prose related to the question, ranking by a mix of vector similarity and literal text match. The winners ride along in the prompt as background. They explain and contextualize; they are never the source of a verified number. When a figure happens to appear in one, the panel reports that as a document occurrence, and the figure's grade does not change.

What it never claims

that anything written in it is a verified fact
that it caused any sentence or figure in the answer (wording overlap is measured separately and is not causation)

document-retrieval vector score

The similarity number that ranked a retrieved context document; a search signal, not confidence and not proof.

Document retrieval embeds the search text and compares it with stored document vectors, blending that similarity with a full-text rank into one ordering score. The score exists to pick which documents ride along as context. It is unrelated to the routing similarity that chose the question family, and the evidence contract states the rule flatly: vectors are retrieval signals, never authoritative quantities.

What it never claims

anything about the truth of the document or of any figure
the same thing as question-routing similarity (different embedding job, different purpose)

Step 4: check answerability

Before any drafting, the gate checks whether the retrieved evidence can support an answer at all. A miss is refused, with the measurements shown instead of an answer.

answerability gate

The pre-drafting check that the question is answerable at all: one relevance signal must clear its floor, and the strict lane separately requires SQL facts.

Before any drafting happens the server asks a cheaper question: can this be answered from what we hold? A confident route, a named person resolving in the data, the question's terms found together in an indexed document, or a retrieval vector near-match can each establish relevance. The strict lane also requires SQL facts for the category. Failing the gate produces a refusal with the measurements shown. Passing it earns the right to draft, nothing more; every figure in the eventual draft is still checked on its own.

What it never claims

that any individual figure is verified (the gate runs before the answer exists)
that clearing a vector floor validated any number

retrieval coverage

A grade on the retrieved context: did the right kinds of document come back, with a measured relevance signal; it never grades figures.

Coverage grades what went in, not what came out. It asks whether the store returned the kinds of document this question type needs and whether some measured signal, such as SQL fact availability, a full-term text match, or a vector near-match, shows the material is actually about the question. Strong coverage means the right material was in front of the model. Whether each published figure checks out is a different, later test.

What it never claims

that any figure in the answer is correct or verified (figure grades are a separate exact-value check)
that the documents' contents are true

Step 5: draft the answer

The writing model receives the question, the retrieved documents, and the SQL fact block, and writes the answer. It has no database access.

SQL fact block

The fact sheet: the SQL facts printed as text lines and placed into the drafting prompt as the model's writing material.

The model never queries the database. Instead the server prints its SQL facts as plain text lines, the fact sheet, and places that block into the drafting prompt. Because the server writes every character of it, the sheet can say where each of its own numbers came from, and it does: a measured value names the stored fact or the arithmetic behind it, and a number that is not a measurement at all says so, such as digits inside a benefit name, the id of the snapshot being reported, or a rank marker in a list. So there is no such thing as a number here with an unknown origin, and appearing on the sheet is not itself a status: a figure earns its status from what accounts for it.

What it never claims

that appearing on this sheet makes a number checked; being printed here is not a status a figure can earn
that the model used a given line to write a given sentence

Step 6: check every figure

After drafting, the server finds every figure in the answer and grades each one against the recorded facts. The grades ship with the answer; nothing is published ungraded without a disclosed reason.

figure status

The per-number outcome of checking a published figure against SQL facts; it describes where the exact value was found, nothing else.

After drafting, server code pulls every number out of the answer and checks each one with plain string and equality comparisons. No model and no similarity search takes part. The result is one status per figure: verified, derived, corroborated, conflict, or no verified source. There is deliberately no status for "it appears in the fact sheet": the fact sheet is written by this server, so it accounts for every number on it, either by naming the stored value behind it or by saying what it is when it is not a measurement at all, such as digits inside a name or a rank marker in a list. A number with no account is simply unverified. The status is a statement about where the exact value was found, and only that.

What it never claims

a single combined confidence scale (routing similarity, retrieval scores, and coverage are separate readouts, not grade inputs)
why the model wrote the number

verified

The exact value matched a named SQL fact seed or a recorded SQL result cell.

Verified is an equality claim: this number equals a value the server itself computed with an approved query against the pinned snapshot, either a named seed or a recorded result cell. It is the strongest status the server can prove, and it is exact relative to the snapshot, not a warranty on the provider's source data.

What it never claims

independent certification of the provider's underlying data

derived

The value was computed from verified values by arithmetic shown alongside it, so a reader can recompute it.

Some numbers an answer needs are never stored anywhere: a participation rate, a gap between two salaries, tenure in years when the store holds months, a total across the rows of a table. Refusing them would refuse ordinary questions, and calling them verified would be false. Derived is the honest third answer: the server publishes the arithmetic beside the figure, naming each operand and its value, so the reader can redo the sum. Both the server's own renderers and the drafting model reach this status the same way, and both attach a disclosure saying the number was computed rather than read.

What it never claims

that the figure is itself a row read from the store

corroborated

An independent cross-check reproduced the value, but no primary named SQL seed exists for it.

Sometimes the server can rebuild a figure a second way even though no registered fact stores it under a name. When the independent computation agrees, the figure is corroborated: two paths reached the same digits. That is meaningful agreement, and it is deliberately not called verified, because no primary stored value stands behind it.

What it never claims

the same thing as a named SQL fact match (corroboration is agreement, not primary provenance)

conflict

The figure was tied to a named fact but disagreed with its stored value; the disagreement is exposed, not smoothed over.

When the model tags a number with a fact name and the stored value disagrees, the server does not pick a winner quietly. The conflict status puts the mismatch on the record so a reader can see both the claimed and the stored value. Depending on guard mode this can also block publication.

What it never claims

which side is right about the world; it reports the mismatch against the snapshot

no verified source

The exact value was found in neither the SQL fact block nor any recorded result cell; the server could not verify it.

This status is an honest shrug with teeth: the server checked everything it trusts and the digits are not there. On the strict path an unexplained untraceable figure does not merely get labeled, it causes the answer to be refused. The status asserts the failure to verify, not the falsity of the claim.

What it never claims

why the model wrote the number, or that the number is false

generated-SQL result

Rows from a generated or raw SQL lane: returned when asked for, always labeled, never interchangeable with registered SQL facts.

An opt-in lane lets a caller run generated or raw SQL. The rows come back marked unverified: they did not pass through the approved query catalog, so they carry none of the registered path's weight. The system's rule for such figures is disclosure rather than silent trust, so the lane's label travels with the data.

What it never claims

registered-path verification (these rows bypassed the approved catalog, the planner, and reconciliation)

What is deliberately not graded

A few numbers are disclosed as outside grading rather than graded, each with a recorded origin.

list numbering

Ordinals the server's own renderer printed, such as the 1. and 2. in front of list items; not claims, not graded, not guarded.

The renderer numbers list items when it lays out the answer. Those ordinals are typography, not claims, so the server records them as exempt at the moment it prints them. They are excluded from both the figure list and the presence guard, because the server, not the model, wrote them.

What it never claims

anything about the answer's content

follow-up figure

A number quoted only in the suggested follow-up questions, never in the answer body; still guarded against fabrication, but not listed as an answer figure.

Suggested follow-ups sometimes mention a number, for example a ratio worth investigating next. Such a figure is guarded like any other, so a fabricated one still blocks publication, but it is not an answer claim, so it gets no figure card and no place in the provenance graph. Guarding a number and listing it as an answer claim are different jobs.

What it never claims

that it is a number the answer used (follow-up suggestions are not part of the answer)

model-noted number

A number the writing model itself flagged with an explanation, published with that note as a disclosure rather than silently trusted.

The drafting contract requires the model to account for every number it writes. When a number is legitimate but not a fact-sheet value, for example a count the model derived while writing, the model attaches a note saying what it is. The server publishes the note as a disclosure. The reader sees the model's own explanation, clearly separated from server-verified facts.

What it never claims

server verification of the value (the note is the model's own accounting)