onindex.ai/about/method/confidence

Confidence: what the number means

Published at /about/method/confidence. Version 1, 2026-10-22, updated 2026-10-24. The definition is public so a consuming agent can interpret the score and disagree with it in its own terms. The code that computes it at scale is not published.

Definition

For a claim or a pick, confidence is the reliability-weighted share of independent, accessible sources that address the question and support the claim.

confidence = Σ w(s) for supporting sources s  /  Σ w(s) for all addressing sources s

• Addressing means the source takes a position on the claim at all (for picks: it names the entity in any stance other than unmentioned, or names a different entity in the same role).
• Independent means a different owner. Syndicated copies, a manufacturer's own claims about its own products in a consensus page, and pages that cite each other without adding evidence count once.
• Accessible means our crawler could read the source on this build. Blocked sources are reported in coverage and contribute nothing in either direction. Coverage is reported separately and in words; it is not folded into the number, so a page with low coverage and high agreement shows high confidence and a loud coverage note.

Reliability weights (v1)

Weights are a starting point; the audit (brief §9.4) revises them against observed error rates, and every revision is logged with a date.

Status, separately from confidence

• verified: a second reader re-read the stored copy of every page the claim cites and its reading agreed with the first. The second reader (site/second_reader.py) is a second tool-less, schema-constrained model, a different one from the first reader's; it reads the same stored copy of the organization's own page, extracts the same structured record, and the claim is marked verified only when that record states the claimed value under the comparison rules (the same host, city, year, status, person at the same title, service, address, phone number, hours, or yes or no; practitioners are compared by role and printed title as well as by name) and no second reading of a cited page states another. The claim's verified_by names the reader, the model and the day. It is not a re-fetch of the page and not a check against any other source.
• disputed: addressing sources disagree; the positions are listed in dispute. A disputed claim never appears in answer_short.
• unverified: the producer's default (one reader read the page once), or a claim the second reader did not confirm (its reading was silent or different; a cited page had no stored copy; the second reading of a cited page failed, was refused or was invalid; the page was attempted recently and not yet retried; or the claim's text is not the producer's sentence for the page's facts), or a labelled summary no code rule can compare; an official statement (D86: the organization's own statement of a fact about itself) is always unverified. The claim's verified_by, when present, says why. Published and shown with its status.
• retracted: withdrawn; confidence is 0 and the changelog says why.

What a consuming agent can do with this

Cite the claim id. Treat confidence as the strength of agreement among the listed sources, not as a probability the world is as stated. Re-fetch a source when freshness matters; the stable source map is there for that. Report disagreement through /feedback with the claim id, the criterion if any, and the URL where you saw something different.