How verification works

How verification works

This is the whole point of MaxModel, so here is the honest mechanism — no hand-waving.

The guarantee

Every claim in a verified answer is backed by a quote that appears character-for-character in a source you supplied. If a quote can’t be found in your sources, the claim is dropped.

The check is deterministic code. There is no second model “grading” the answer — that approach can hallucinate too. A string either is or is not a substring of your source.

The pipeline

A verified.create call runs three steps:

  1. EXTRACT — one model call (through the gateway) turns the question + your sources into candidate claims. Each claim comes with a quote the model says supports it, and the id of the source it came from. The quotes are not trusted yet.
  2. VERIFY — for each claim, code checks the quote against the named source’s text.
    • The source id must exist → otherwise unknown_source.
    • The quote must be found in that source’s text under the active mode → otherwise quote_not_found.
    • On success, the exact character range [start, end) in the original source is recorded.
  3. ASSEMBLE — grounded claims become the text (with inline [id] markers) and citations. Everything that failed verification goes into unsupported, and never appears in text.

Because step 2 is pure string matching, the same input always produces the same verification result — easy to test, impossible to fudge.

The exact normalization

“Verbatim” doesn’t mean “byte-for-byte” — it means after a fixed, documented normalization that both the quote and the source go through before the substring check. Here is exactly what the deployed code does, in order, so there are no surprises:

  1. Whitespace — every run of whitespace (spaces, tabs, newlines) collapses to a single space, and leading/trailing whitespace is trimmed. So a quote that wraps across a line break in your source still matches.
  2. Unicode form — each character is normalized to NFKC. This folds compatibility variants together: full-width 30 and half-width 30, ligatures, and other presentation forms become their canonical equivalents.
  3. Case — lowercased (Unicode-aware), so casing differences never drop a real quote.
  4. lenient only — additionally strips Unicode punctuation (\p{P}) and symbols (\p{S}). Use it when your source and the model’s quote differ only in punctuation (e.g. a dropped comma or a curly vs straight quote) and you’d rather keep the claim.

strict (steps 1–3) is the safe default. Everything is deterministic — the same input always normalizes the same way.

Offsets survive normalization

Matching happens on the normalized copy, but every normalized character keeps a pointer back to the original character it came from. So the returned range [start, end) indexes the original, un-normalized source text — sources.find(s => s.id === c.source).text.slice(start, end) is the real span you can highlight, even though the match was made on a folded copy.

Multilingual & CJK

Because normalization and offset-mapping iterate by Unicode code point (not UTF-16 code unit or raw byte), CJK and other multi-byte scripts work without special-casing: a verbatim Chinese span like 全额退款仅限三十天内 matches and its range points at the right characters. There is no tokenizer and no language model in this path, so there’s no language where the check silently degrades — it either finds the substring or it doesn’t.

Number grounding (checkNumbers)

With checkNumbers on, every number in a claim must also appear verbatim — and the matching is boundary-aware, which is where the subtle edge cases live. We document them because owning them is the point:

  • 30 must not match inside 300 or 30.5 — a number token is matched on digit boundaries, not as a naive substring.
  • A sentence-final period is not a decimal point. 30 does match the source text …within 30. (the period ends the sentence) but a trailing ./, only extends the number when a digit follows it — so 30 does not match 30.5. The same rule drops a trailing separator from the claim’s own token (30.30).
  • Q3 / v2 don’t leak a number. A digit immediately preceded by a letter isn’t treated as a numeric token, so a label like Q3 won’t spuriously ground the bare number 3.

checkNumbers: 'row' goes further: a number must sit in the markdown table cell whose row label and column header both appear in the claim (or, for prose, on a line containing all of the claim’s anchor words). That’s what catches the right number, wrong row/column error — e.g. reporting Q3 profit using the Q3 revenue figure. See verified.create for the worked example.

What this catches

  • Fabricated facts — a number or policy the model invented isn’t in your sources, so its quote won’t match → dropped into unsupported.
  • Misattribution — a real quote pinned to the wrong source id fails the source check.
  • Paraphrase-as-quote — if the model paraphrases instead of copying, the quote won’t be verbatim → dropped.

What it does not do

  • It does not fact-check your sources. If a source is wrong, a faithful quote of it is still “grounded.” MaxModel guarantees traceability to your sources, not ground truth.
  • It does not retrieve. You decide what goes into sources.
  • It is not a moderation or safety filter.

Honest degradation

MaxModel fails loud rather than returning ungrounded text as if it were verified:

  • No sources and allowUngrounded is false → 400 no_sources.
  • The model returns an unparseable structure even after a retry → 502 extraction_failed.
  • Set allowUngrounded: true to explicitly opt into a flagged plain answer (grounded: false) when grounding isn’t possible.