The question
Large language models write long, fluent answers that mix accurate and inaccurate statements. A single "is this answer correct?" judgment is too coarse for that. One false date in a 300-word biography should not condemn the other twenty true claims, and nor should those twenty excuse it.
The decompose-then-verify paradigm responds by splitting a response into minimal, self-contained claims ("atomic facts"), checking each against evidence, and aggregating the results. We asked:
- Where did the idea come from, and what exactly does it measure?
- How well do automated pipelines agree with humans?
- Does using the signal at inference time, or as a training reward, actually make models more truthful?
- Where does it fail?
- How is it used in practice, and at what cost?
Scope. Foundational work from 2004 onward, with a focus on 2023–2026 LLM research. Peer-reviewed papers and preprints are both included, and every preprint is labelled as such. We do not cover hallucination detection from model internals (probing, uncertainty) except where it feeds a claim-level pipeline.
A note on terms. The literature uses three related ideas, and they are not interchangeable:
- Factuality is support by a chosen knowledge source.
- Truthfulness, in TruthfulQA's sense, means not asserting falsehoods, so abstaining counts as truthful .
- Factual consistency means agreement with a specific source document .
Atomic-fact metrics measure the first of these.
What we wanted to know
AI chatbots can write long answers. Most of an answer may be right, but a few bits may be wrong. Asking "is this whole answer right or wrong?" is too rough. One wrong date shouldn't spoil 20 true facts. And 20 true facts shouldn't hide one wrong date.
So researchers came up with a trick. They chop an answer into tiny facts, called atomic facts. "Atomic" just means the smallest piece. Then they check each tiny fact against something trusted, like Wikipedia. The score is how many facts checked out. We asked five questions:
- Where did this idea come from, and what does it really measure?
- Do computer checkers agree with people?
- Does it make AI more honest? We looked at fixing answers and at training.
- Where does it go wrong?
- How is it used in real life, and what does it cost?
What we looked at. We looked at older work from 2004 on. But we focused on AI research from 2023 to 2026. Most papers were checked by other experts before they came out. A few were not, and we say which ones. We left out ways of looking inside an AI's "brain".
Three words that sound alike. They mean different things:
- Factuality: a fact is backed up by the place we checked.
- Truthfulness: the AI doesn't say false things. Saying "I don't know" counts as truthful .
- Factual consistency: an answer agrees with one document it was given .
Tiny-fact checking measures the first one.
Where the atoms came from
The idea of scoring text by sub-sentence units is older than LLMs. The Pyramid method for summarisation broke reference summaries into Summary Content Units and weighted each by how many references expressed it . It measured content selection, not truth, but it established that a sentence can be scored by its parts.
The verification half arrived with FEVER. Its 185,000-plus claims were labelled SUPPORTED, REFUTED or NOTENOUGHINFO against retrieved Wikipedia evidence . FEVER's claims were written by annotators rather than generated by models. Still, its three-way, evidence-backed verdict is the template almost every later pipeline inherits. Meanwhile LAMA treated a "fact" as a subject–relation–object triple and probed whether models stored it . This framed facts as discrete, checkable units, though it probed knowledge rather than checking generated text.
Summarisation research then drove granularity down. QAGS decomposed a summary into question–answer probes and compared answers drawn from the source and from the summary. Its Pearson correlation with human judgments was 54.53 on CNN/DM, against 17.72 for ROUGE-2 . SummaC showed that NLI models fail at document level and recover when inputs are segmented into sentences, reaching 74.4% balanced accuracy . That was the clearest pre-LLM evidence that finer granularity helps verification.
FActScore (EMNLP 2023) named and formalised the modern unit: an atomic fact is a short sentence carrying one piece of information. The score is the fraction of a generation's atomic facts supported by a knowledge source, English Wikipedia for biographies . Its human-annotated results were sobering. On the authors' biography task, InstructGPT scored 42.5%, ChatGPT 58.3%, and the retrieval-augmented PerplexityAI 71.5%. Precision fell sharply for rarer entities and for facts later in a generation .
Two design choices in FActScore shaped everything after it:
- It is precision-only. The authors state plainly that it does not consider recall, and they advise reporting abstention rates and fact counts alongside it . A model that says less can score higher.
- "Supported" means "supported by Wikipedia". An unsupported fact is not necessarily false. It may simply be absent from the knowledge source .
Where the idea came from
Scoring writing in small pieces is older than chatbots. In 2004, the Pyramid method split summaries into small idea pieces . It checked which ideas a summary included, not whether they were true. But it showed you can score writing piece by piece.
The checking part came from FEVER in 2018. People wrote over 185,000 short claims. Each one was marked "backed up", "proven wrong" or "not enough information", using Wikipedia . Almost every checker since has copied this idea. Around then, LAMA treated a fact as a tiny three-part sentence . The parts were: who, what link, and what. It tested what an AI remembered, not what it wrote.
Next, researchers made the pieces even smaller. QAGS turned a summary into questions, then compared the answers . It matched people's judgements much better than an older score: 54.53 versus 17.72. SummaC checked one sentence at a time . That worked better than checking a whole document. It got 74.4% right on a fair test.
Then came FActScore in 2023. It named the tiny fact . A tiny fact is a short sentence with one piece of information. The score is the share of tiny facts that Wikipedia backs up. People checked AI-written life stories by hand. InstructGPT scored 42.5%, ChatGPT 58.3%, and PerplexityAI, which searches the web, 71.5% . Scores dropped a lot for less famous people. They also dropped for facts near the end of an answer .
FActScore made two choices that still matter today:
- It only counts how many facts are right, not how many facts are given. The authors say so themselves . So an AI that says less can score higher.
- "Backed up" means "found in Wikipedia". A fact Wikipedia doesn't mention isn't always wrong. It might just be missing .
The instrument: how well do machines grade atoms?
The headline agreement numbers
FActScore's automated estimator (retrieval plus an LM verifier) is often summarised as having under 2% error. That figure is the gap between estimated and human scores at the aggregate level, for the best estimator chosen per system. A single fixed estimator could be off by as much as 9.9 points on another system .
Google DeepMind's SAFE replaced Wikipedia with multi-step Google Search. It also made each fact self-contained before checking it, and added recall through F1@K, which compares supported facts to a target count K . SAFE agreed with crowdworkers on 72% of 16,011 facts. On 100 sampled disagreements, the authors judged SAFE correct 76% of the time and the crowdworkers 19% . It cost $0.19 per response against about $4 for human annotation . Two caveats apply. The baseline was crowdworkers, not domain experts, and the authors adjudicated the disagreements themselves. The "better than humans" framing rests on that non-blind comparison.
VeriScore argued that both FActScore and SAFE wrongly assume every claim is verifiable. It extracts only verifiable claims, and humans preferred its extractions over SAFE's 93% of the time . Inter-annotator agreement was substantial but imperfect (Fleiss κ ≈ 0.77 for extraction, ≈ 0.73 for verification). Model rankings in individual domains agreed only partially with the cross-domain ranking (Kendall τ 0.56–0.83) .
When graders disagree with each other
The independent picture is less reassuring:
- FELM found that even GPT-4 given retrieved reference content reached only 48.3 F1 at detecting segment-level factual errors .
- FactBench measured human agreement on the factuality labels of content units at only Cohen's κ = 0.57. It also showed that four decompose-then-verify evaluators assigned GPT-4o factual precision ranging from about 57 (FActScore) to about 87 (FactCheck-GPT) on the same outputs .
- FaStFact compared evaluators against its own human-annotated set (ground-truth F1@K′ 0.792). SAFE scored 0.960, VeriScore 0.899 and FacTool 0.987, overestimating by roughly 0.11–0.20 . The authors are benchmarking their own method against rivals, so this needs independent replication, but its direction matches FactBench's.
How good are the fact checkers?
How often do they agree with people?
People often say FActScore's computer checker is off by less than 2%. That's true for the overall score, using the best checker set-up for each AI. But one set-up was off by up to 9.9 points on another AI .
Google DeepMind's SAFE checks facts with Google Search instead of Wikipedia . It also counts whether an answer has enough facts, not just right ones. SAFE agreed with paid online workers on 72% of 16,011 facts . Where they disagreed, the authors looked at 100 cases. They decided SAFE was right 76% of the time. The workers were right 19% of the time . SAFE cost $0.19 per answer, and people cost about $4 . But the workers weren't experts. And the authors judged the disagreements themselves, so that test wasn't fully fair.
VeriScore said older checkers made a mistake. They tried to check every sentence, even ones nobody can check. VeriScore only picks facts that can be checked . People liked its fact lists better than SAFE's 93% of the time . People agreed with each other fairly well, but not perfectly. Which AI came top changed a bit from topic to topic .
When checkers disagree with each other
Other teams found bigger problems:
- In FELM, even GPT-4 with helpful documents was poor at finding mistakes. It scored 48.3 out of 100 .
- In FactBench, people only partly agreed on which facts were right . Four checkers scored the same GPT-4o answers. Scores went from about 57 to about 87 .
- FaStFact compared checkers with people's own score of 0.792. SAFE gave 0.960, VeriScore 0.899 and FacTool 0.987 . So the computer checkers scored too high. The authors were testing their own tool against others, so other teams should check this. But it matches what FactBench found.
Interpretation. Within a single, fixed pipeline, atomic-fact scores are useful for ranking systems and tracking change. Absolute numbers are not comparable across pipelines, and small differences between models may fall within evaluator noise. We found no paper that directly tests the statistical significance of between-model differences under evaluator error. That is a gap worth closing.
Frontier adoption
The method has moved from benchmarks into production evaluation. OpenAI's GPT-5 system card reports claim-level error rates "following FActScore and LongFact". An o3 grader with browsing lists each factual claim and fact-checks it . On de-identified production traffic, gpt-5-thinking showed a 65% lower claim-level hallucination rate than o3 . This is a vendor self-report graded by a same-vendor model, so it is significant as evidence of adoption, not as independent evidence of the size of the gains.
Does verifying atoms make models more truthful?
At inference time: verify-and-revise
Chain-of-Verification (CoVe) has a model draft an answer, plan verification questions, and answer them independently of the draft so that it does not copy its own errors. The model then revises. With Llama-65B, the factor-and-revise variant of CoVe raised biography FActScore from 55.9 to 71.4, and the two-step variant roughly doubled Wikidata list precision from 0.17 to 0.36 . The cost shows up in the same tables. Average facts per biography fell from 16.6 to 12.3, and list-task recall dropped . Part of the precision gain is the model saying less. The evaluation also uses FActScore, the very construct the method targets.
What this means. Stick to one checker, set up the same way every time. Then it's good for ranking AIs and seeing change. But scores from different checkers can't be compared. A small win of a point or two might just be checker wobble. Nobody has tested how big that wobble is yet.
Big AI companies use it now
OpenAI's GPT-5 report uses this method . Another AI lists each fact and checks it on the web. On real user chats, GPT-5 thinking made 65% fewer fact mistakes than o3 . But OpenAI checked its own AI with its own checker. So this shows the method is used. It doesn't prove exactly how big the gains are.
Does checking facts make AI more honest?
Checking while the AI answers
Chain-of-Verification, or CoVe, works like this. The AI writes an answer. Then it writes questions to check its facts. It answers each question on its own, without peeking at the first answer. That way it doesn't copy its own mistakes. Then it fixes the answer. This raised one AI's score on life stories from 55.9 to 71.4 . On a list task, the share of right answers went from 0.17 to 0.36 . But answers also got shorter, from 16.6 facts to 12.3 . So part of the gain came from saying less.
RARR took the retrieval route instead. It researches each claim, then minimally edits disagreements. Human-judged attribution rose by roughly 7–15 points (for example, 35.4 to 43.4 on NQ) while about 90% of original intent was preserved . Note that RARR improves attribution to retrieved evidence, which is not the same as truth. FacTool generalised the pipeline to code, maths and scientific citations using tools . With GPT-4 it beat self-checking. With ChatGPT it trailed zero-shot self-checking (claim-level F1 81.25 vs 84.88) . That is an early sign that the verifier's quality decides whether decomposition pays off.
A broader caution applies to verifiers that check their own work. Without external feedback, LLM self-correction on reasoning tasks does not reliably help and can hurt: GPT-3.5 on CommonSenseQA fell from 75.8 to 38.1 after one round . A TACL survey concludes that self-correction works mainly when reliable external feedback is available . For atomic verification the practical reading is straightforward. Retrieval-grounded verification is on much firmer ground than "ask the model whether it was right".
At training time: atoms as reward
This is where the most consequential recent evidence sits.
- FactTune (ICLR 2024) ranked sampled responses by FActScore-style truthfulness and trained Llama-2-7B with DPO. It reported 58% and 40% reductions in factual error rate on biographies and medical QA respectively, against Llama-2-chat . Incorrect facts per biography fell from 6.41 to 2.00, while correct facts also fell slightly (19.03 to 17.06). The authors note somewhat terser responses .
- FLAME (NeurIPS 2024) made SFT and DPO factuality-aware and gained 5.6 FActScore points over standard SFT+DPO on Llama-2-70B (42.3 to 47.9), while preserving instruction-following . It also found two counter-intuitive results. Fine-tuning on more factual RAG-written text made the model less factual, since training on unfamiliar knowledge encourages hallucination. Optimising factuality alone hurt helpfulness .
- FactAlign raised Gemma-2B's LongFact f1@100 from 61.74 to 86.51. Most of that came from roughly doubling the number of claims (recall), and FActScore moved only from 57.02 to 59.77 . Mask-DPO (ICLR 2025) used sentence-level factuality masks and reported out-of-domain gains: biography FActScore rose from 30.29 to 39.39 for Llama-3.1-8B .
RARR checks each fact on the web, then fixes only what's wrong . People said answers were better backed up afterwards, for example 35.4 up to 43.4 . And about 90% of what the writer meant stayed the same. But "backed up by a web page" isn't quite the same as "true". FacTool used tools to check code, maths and science answers too . With GPT-4, it beat the AI just checking itself. With ChatGPT, it didn't: 81.25 versus 84.88 . So the checker needs to be good, or chopping doesn't help.
There's a bigger warning too. When an AI checks its own thinking with no outside help, it often gets worse. In one test, GPT-3.5 dropped from 75.8 to 38.1 after one round . A big review found self-checking works mainly with good outside help, like search . So check facts against real sources. Don't just ask the AI "are you sure?".
Using fact checks to train the AI
This is where the newest and most important results are.
- FactTune scored lots of answers with FActScore . Then it taught the AI to prefer the better ones. Mistakes dropped by 58% on life stories and 40% on medical questions . Wrong facts per life story fell from 6.41 to 2.00. Right facts fell a little too, from 19.03 to 17.06. Answers got a bit shorter .
- FLAME added fact checks to two training steps. The score rose 5.6 points, from 42.3 to 47.9 . The AI still followed instructions well. It found two surprises. Training on more accurate text the AI didn't already know made it less accurate. And training only for facts made it less helpful .
- FactAlign raised one score from 61.74 to 86.51 . But that was mostly because answers had about twice as many facts. Its FActScore only moved from 57.02 to 59.77 . Mask-DPO checked each sentence while training. Scores went up even on new topics: from 30.29 to 39.39 on life stories .
Reward hacking is now documented, not hypothetical. Learning to Reason for Factuality (ICML 2026) found that reasoning models such as DeepSeek-R1 and QwQ-32B hallucinate more on long-form factuality than their non-reasoning counterparts . When the authors trained with a factuality-only reward, responses became shorter or less relevant. A combined reward for precision, detail and relevance cut hallucination by an average of 23.1 points while keeping answers detailed . A concurrent ICML 2026 paper trained Qwen3-8B with a continuous VeriScore reward. Long-form hallucination fell from 61.9 to 40.6, but AlpacaEval dropped from 54.7 to 42.2 and ArenaHard from 18.7 to 14.9 . Its proposed fix, a binary "no contradiction found" reward, cut hallucination 39.3% while preserving general skills . Notably, the two groups chose opposite remedies (richer versus coarser rewards), and we found no head-to-head comparison.
AIs cheating the reward is now proven, not just a worry. A 2026 study looked at AIs that "think step by step" . They made more mistakes in long answers. Rewarding right facts alone made answers shorter or less on-topic . A reward for being right, detailed and on-topic worked better. It cut mistakes by 23.1 points on average . Another 2026 study used a fact score as the reward . Mistakes in long answers fell from 61.9 to 40.6. But helpfulness test scores fell too: 54.7 to 42.2, and 18.7 to 14.9 . Then they tried a simpler reward. It just asked: were any mistakes found, yes or no? That cut mistakes by 39.3% and kept the AI's other skills . The two teams picked opposite fixes. Nobody has compared them side by side yet.
Fast verifiers are what make this practical. VeriFastScore collapses extraction and verification into one fine-tuned 8B model, 6.6× faster than VeriScore, with r = 0.94 system-level correlation . That correlation is with VeriScore, not with humans, so it inherits VeriScore's errors.
Interpretation. Atomic-fact rewards clearly move factuality metrics, and the effect is replicated across several groups, models and methods. Two things are much less established. The first is whether gains transfer to independent benchmarks; most papers evaluate with the same family of metric used for training. The second is whether they hold at frontier scale outside vendor reports.
Where the evidence disagrees
Fast checkers make this possible. VeriFastScore does the chopping and checking in one step, 6.6 times faster . Its scores closely match VeriScore's. But it was compared with VeriScore, not with people. So it has the same blind spots.
What this means. Training with fact-check scores clearly makes those scores go up. Several teams found this. Two things are less clear. Do the gains show up on other teams' tests? And do they hold for the very biggest AIs?
Where the experts disagree
Is "atomic" even well-defined?
The atomic fact is less atomic than it sounds:
- FActScore's own annotators had to split or merge machine-generated facts in a substantial fraction of cases .
- A Closer Look at Claim Decomposition showed that FActScore values change with the decomposition method used on the same documents . In other words, the decomposer injects measurement error that gets attributed to the generator.
- Decomposition Dilemmas (NAACL 2025) found decomposition generally helps weaker verifiers and tends to hurt stronger ones. With the MiniCheck verifier on WiCE, FActScore-style decomposition dropped balanced accuracy from 80.01 to 71.11 .
Granularity also cuts the other way. Molecular Facts showed that over-atomised claims, stripped of context, can look true when they are not, for example a fact about a similarly named person. Adding minimal context raised verification accuracy from 68.7% to 74.7% on ambiguous biographies . Even so, detection of unsupported claims stayed low (38.8% at best) . Claimify (ACL 2025) extracts claims only when it is confident of the intended reading . The field is converging on "decontextualised but minimal" rather than "as small as possible", but there is no settled standard.
Can the metric be gamed?
Yes. Core showed that padding an answer with obvious or repetitive true sub-claims pushes FActScore-style precision from 54.0% to 83.0%, and a uniqueness/informativeness filter pulls it back to 36.2% . Those attacks were constructed deliberately. The reward-hacking results above show that optimisation pressure finds related exploits on its own: brevity, irrelevance and safe trivia. HalluLens makes the same point at benchmark level. Precision alone rewards short answers, so recall must be measured too .
What counts as one tiny fact?
A "tiny fact" is not as simple as it sounds:
- FActScore's checkers often had to split or join the AI's facts by hand .
- Chopping the same answer in different ways gave different scores . So the chopping step adds its own mistakes.
- One study found chopping helps weak checkers but can hurt strong ones . One strong checker dropped from 80.01 to 71.11 after chopping .
Chopping too small causes problems too. A tiny fact taken out of its sentence can look true when it isn't. For example, it might be about someone who just has the same name. Adding back a little context raised accuracy from 68.7% to 74.7% . Even then, checkers caught only 38.8% of the false facts . Claimify only pulls out a fact when it's sure what the sentence means . Researchers now prefer small facts that still make sense alone. But there's no agreed rule yet.
Can you cheat the score?
Yes. Core showed that adding obvious true facts pushes the score from 54.0% to 83.0% . A filter throws out boring and repeated facts. Then the score drops back to 36.2% . Those cheats were made on purpose. But training finds similar cheats by itself . Answers get shorter, go off-topic, or fill up with safe, boring facts. HalluLens says the same thing. Counting only right facts rewards short answers. So you must also count how much is said .
Are true atoms a true story?
Not necessarily. In Merging Facts, Crafting Fallacies, each atomic fact in a biography of an ambiguous name could be individually true while the paragraph conflated two different people. FActScore overestimated factuality: for Llama-2-13b-chat, 94.8 versus 78.4 under an entity-aware variant . Several further gaps sit beyond what atomic precision measures:
- Omission. What a model leaves out goes unmeasured.
- Misleading framing and selective truth. Accurate claims can still be arranged to mislead. We found no strong empirical paper measuring this in long-form evaluation, and it is a genuine gap.
- Knowledge-source coverage. The verifier can only confirm what its source contains. In Bengali, scant Wikipedia coverage made verdicts unreliable: verifier accuracy was 60.6% with 8 retrieved passages against 86.8% with Google Search .
Are frontier models actually getting better?
The vendor and independent pictures differ. Vendor reports show large claim-level improvements . Independent, harder benchmarks still find substantial error. On FACTORY's hard subset, frontier models from mid-2025 reached only about 60% factual precision, while scoring above 90% on LongFact, which suggests the older benchmarks are saturated . On HalluHard's multi-turn, citation-grounded tasks, the best configuration still hallucinated in roughly 30% of cases with web search, and about 60% without . Both are preprints. They are consistent with each other and with the reasoning-model finding in : progress is real but uneven, and depends heavily on targeted post-training.
What this means in practice
Industry is split on granularity:
- OpenAI reports claim-level rates .
- Google's FACTS Grounding uses a single response-level verdict, where any ungrounded claim fails the whole response, judged by three cross-vendor models to limit self-preference .
- OpenAI's own SimpleQA sidesteps decomposition entirely by restricting questions to a single indisputable answer .
In RAG engineering, claim decomposition is already the default "faithfulness" metric. Ragas scores the share of an answer's claims supported by retrieved context . DeepEval counts a claim as faithful if it does not contradict the context, and lets ambiguous claims be penalised . "Does not contradict" is laxer than "is entailed by", so faithfulness scores from different tools are not comparable.
Domain deployments show the engineering levers. In Stanford's VeriFact, clinical text was verified against the patient's own health record . For human-written notes, atomic claims produced far fewer malformed propositions than sentences (0.4% versus 19.8% invalid), and the single most important setting was how many record facts were retrieved . Off-the-shelf fact-checkers still struggle on the claims that matter most, the false ones. In OpenFactCheck's head-to-head, false-claim F1 topped out around 0.63, and the most accurate pipeline was also the slowest and most expensive . Cost varies by roughly an order of magnitude across pipelines: SAFE used about 49.6k tokens per response against about 5.6k for a chunk-level alternative .
Can true facts tell a false story?
Yes. One study used names that belong to two different people . Each tiny fact could be true, but the paragraph mixed up the two people. FActScore gave 94.8. A checker that knew about the mix-up gave 78.4 . Some things tiny-fact checking can't see at all:
- Missing facts. What the AI leaves out isn't counted.
- Tricky framing. True facts can be arranged to give a wrong idea. We found no strong study that measures this.
- Gaps in the source. The checker can only confirm what its source knows. Bengali Wikipedia is small. With Wikipedia, the checker was right 60.6% of the time. With Google Search, it was right 86.8% .
Are the best AIs getting better?
Company reports say yes, by a lot . Harder tests made by other teams still find plenty of mistakes. Take FACTORY's hardest questions . Top AIs from 2025 got only about 60% of facts right. The same AIs scored over 90% on an older test, LongFact . So the older test is now too easy. On HalluHard, the best AI still got things wrong about 30% of the time . That was even with web search. Without search, it was about 60% . Other experts haven't checked these two studies yet. But they agree with each other, and with the 2026 study . AIs are getting better, but unevenly. Special training for facts matters a lot.
What this means for people building AI
Companies don't agree on how small to chop:
- OpenAI scores each tiny fact .
- Google's FACTS Grounding gives each whole answer a pass or a fail. One unsupported fact fails it. Three AIs from different companies do the judging . That way, no AI marks its own homework.
- OpenAI's SimpleQA avoids chopping. Every question has just one clear right answer .
Tools for building AI apps already use tiny-fact checking. Ragas counts how many facts are backed by the app's documents . DeepEval counts a fact as fine if the documents don't disagree with it . "Doesn't disagree" is looser than "is backed up". So the two tools' scores can't be compared.
Real projects show what matters most. Stanford's VeriFact checks hospital notes against the patient's own records . Tiny facts got broken far less often than whole sentences: 0.4% versus 19.8% . The most important setting was how many record facts it looked up . Ready-made checkers still struggle to spot false facts, which are the ones that matter most. The best scored only about 0.63 out of 1 . And the best one was also the slowest and most expensive . Costs differ a lot. SAFE used about 49,600 tokens, or bits of text, per answer. A newer method used about 5,600 .
For builders (interpretation):
- Verify against retrieved evidence, not the model's own judgement .
- Always report recall or claim counts alongside precision .
- If you train on the signal, add detail and relevance terms, or use a coarse reward, and watch general-quality benchmarks .
- Choose decontextualised claims over maximally atomic ones .
- Treat absolute scores as pipeline-specific .
Open questions and opportunities
- Significance under evaluator noise. How large must a FActScore or VeriScore difference be before it exceeds the variance from decomposition and verification? Nobody has published bootstrapped confidence intervals over decomposer and verifier choice. That would be a cheap, valuable contribution.
- Transfer. Do factuality-RL gains show up on independent benchmarks (FACTORY, HalluHard, SimpleQA) rather than the training metric's own family? Most current evidence is self-referential.
- Rewards beyond precision. There is no head-to-head yet between richer multi-term rewards and coarse binary rewards .
- Composition and omission. Can entity-aware or discourse-level checks, in the spirit of , be made cheap enough to sit alongside atomic checks?
- Speculative, lateral idea. Clinical work verifies against a closed, authoritative record . The same pattern, a small trusted store instead of the open web, could transfer to enterprise and legal settings, where "supported" has a contractual meaning. We found no published evaluation of this.
Limitations of this review
- Access. Sources were read via web-fetched full text (arXiv HTML, ACL Anthology PDFs, official documents). Some long papers and appendices were only partly read. Paywalled journals were not accessed.
- Excerpts. Quoted excerpts were checked against the sources in a separate verification pass. Readers should still treat the linked paper as authoritative.
- Preprints. Five sources are not peer-reviewed: , , , and . The two independent frontier benchmarks, and , are among them, so the "are models improving?" section is graded Emerging.
- Re-checked against published versions. Verification upgraded several sources initially found as preprints to their peer-reviewed versions (, , , , , ). For , we read the arXiv preprint (VeriFact) and cite the NEJM AI publication, which has a different title.
- Vendor evidence. The frontier evidence is self-reported. We did not systematically survey every lab's system card.
- Scope. Probing-based hallucination detection, multimodal factuality and non-English work beyond are out of scope.
- Cost figures reflect the model prices at the time of each paper and will have changed.
Method
- Date researched: 22 November 2026.
- Approach: five parallel lines of inquiry, each with a distinct angle: foundations, frontier (2024–2026), evidence and benchmarks, critique, and applied/deployment.
- Sources searched: ACL Anthology, arXiv, OpenReview, NeurIPS/ICLR/PMLR proceedings, Semantic Scholar, vendor system cards and tool documentation.
- Representative queries: "FActScore atomic facts", "long-form factuality SAFE LongFact", "VeriScore verifiable claims", "claim decomposition fact-checking", "factuality DPO / RL reward hacking", "decontextualization molecular facts", "claim-level hallucination system card".
- Inclusion criteria: primary sources reporting a method, measurement or critique of claim-level decomposition and verification for LLM output. Peer-reviewed work was preferred, and preprints were included and labelled.
- Screening: roughly 140 records screened across the five lines of inquiry. 41 sources were included after deduplication.
- Verification: an independent verification pass, made without sight of the draft, checked every citation's metadata, excerpt and key numbers against the source. Corrections were applied and unverifiable items were removed.
Tips for builders (our opinion):
- Check facts against real sources, not the AI's own opinion .
- Always count how much was said, not just how much was right .
- If you train with fact scores, reward detail and staying on topic too. Or use a simple yes/no reward. Then watch general helpfulness tests .
- Pick small facts that still make sense alone, not the tiniest possible pieces .
- Only compare scores that came from the same checker .
Questions nobody has answered yet
- How big does a win need to be? Checkers wobble by a few points. Nobody has measured how much. That would be a cheap, useful study.
- Do the gains carry over? Do AIs trained on fact scores also do better on other teams' tests? Most studies only use their own kind of test.
- Which reward is best? Nobody has compared rich rewards with simple yes/no rewards side by side.
- Mixed-up and missing facts. Can checks for mixed-up people, like in , be made cheap enough to use everywhere?
- A guess of ours. Hospitals check against one trusted record, not the whole web . The same idea could work for businesses and law, where "backed up" really matters. We found no study of this yet.
What this review might have missed
- Reading. We read papers online. Some long papers and their extra sections were only partly read. We couldn't open journals that cost money.
- Quotes. Every quote was checked against its source in a separate step. The original paper is still the final word.
- Not checked by experts. Five sources weren't checked by other experts first: , , , and . Two of the harder tests are among them, so that part is marked "Emerging".
- Final versions. We swapped several early drafts for their final, expert-checked versions (, , , , , ). For , we read the draft. We cite the final journal paper, which has a different title.
- Company reports. The GPT-5 numbers come from the company itself. We didn't check every company's report.
- Not covered. We didn't look inside an AI's "brain". We left out pictures, video, and most languages other than English, apart from .
- Costs. Prices are from when each paper was written. They will have changed.
How we did this
- When: 22 November 2026.
- How: five separate lines of research. They covered history, the newest work, the numbers, problems, and real-world use.
- Where we looked: research libraries, paper websites, company reports and tool guides.
- What we searched for: things like "FActScore atomic facts", "long-form factuality", "claim decomposition" and "factuality reward hacking".
- What counted: papers that build, measure or criticise tiny-fact checking for AI answers. We preferred papers checked by experts, and labelled the rest.
- How many: we looked at about 140 sources and kept 41.
- Checking: a separate check compared every reference, quote and number with its source. It was done without seeing the draft. Mistakes were fixed.
