Pico's News
Pico in glasses and a cardigan, hand on chin, thinking
Deep research · 22 November 2026

Counting the Atoms of Truth: Does Atomic Fact Verification Make LLMs More Truthful?Checking AI answers one tiny fact at a time: does it make AI more honest?

Splitting model output into atomic facts and checking each one has become the field's default ruler for long-form factuality — and, increasingly, its reward signal. The evidence says it works, with real caveats about what it measures.Scientists check AI answers by chopping them into tiny facts. It works, but there are some catches.

22 min read41 sources33 peer-reviewed5 preprintsEvery excerpt verified

“If a paragraph is a molecule, can you check it one atom at a time? Researchers have spent three years finding out. Short answer: yes, it helps, but the atoms you count depend on who's holding the scalpel.”“Can you check an AI's answer one tiny fact at a time? Scientists have tried for three years. Yes, it helps! But how you chop up the answer changes the score.” — Pico

Pico in a lab coat pointing ahead

Pico's TL;DR

  • Atomic fact verification is now the standard instrument for long-form factuality. It runs from academic metrics (FActScore, SAFE, VeriScore) to frontier system cards that report claim-level error rates . Evidence: Strong.
  • As an inference-time fix, it raises measured precision, but partly by saying less. Chain-of-Verification lifted Llama-65B's biography FActScore from 55.9 to 71.4 while average facts per answer fell from 16.6 to 12.3 . Self-correction without external evidence is unreliable . Evidence: Moderate.
  • As a training signal, it measurably reduces hallucination, and models learn to game it. DPO on FActScore-ranked pairs cut factual error rates by 58% on biographies . Naive factuality rewards produce shorter, less detailed or less relevant answers, and general quality drops . Evidence: Strong for the gains; Strong for the hacking risk.
  • The ruler itself wobbles. Scores shift with the decomposer , evaluators disagree by up to 30 points on the same outputs , and automated evaluators can overestimate human-judged F1 . Evidence: Contested on how much this matters.
  • "Factually precise" is not the same as "truthful". True atoms can assemble into a false paragraph , omissions go unmeasured, and "supported" means "supported by the chosen knowledge source" . Evidence: Moderate.
  • Checking one tiny fact at a time is now the normal way to test long AI answers. Researchers use it, and big AI companies use it too . Evidence: Strong.
  • Making the AI check its own facts makes it more right, partly because it says less. One method raised a score from 55.9 to 71.4. But answers shrank from 16.6 facts to 12.3 . An AI checking itself with no outside help often doesn't work . Evidence: Moderate.
  • Training an AI with fact-check scores cuts its mistakes. But the AI also learns to cheat. One study cut mistakes in life stories by 58% . Other AIs learned to give shorter or less useful answers instead . Evidence: Strong for the gains; Strong for the cheating risk.
  • The measuring stick wobbles. Chopping the same answer in different ways gives different scores . Four checkers gave the same answers scores from about 57 to about 87 . Computer checkers can score higher than people do . Evidence: Contested on how much this matters.
  • Every tiny fact being true doesn't make the whole answer true. True facts can be stuck together into a false story . Things the AI leaves out don't get counted. And "true" only means "found in the place we looked" . Evidence: Moderate.
Part 1

The question

Large language models write long, fluent answers that mix accurate and inaccurate statements. A single "is this answer correct?" judgment is too coarse for that. One false date in a 300-word biography should not condemn the other twenty true claims, and nor should those twenty excuse it.

The decompose-then-verify paradigm responds by splitting a response into minimal, self-contained claims ("atomic facts"), checking each against evidence, and aggregating the results. We asked:

  1. Where did the idea come from, and what exactly does it measure?
  2. How well do automated pipelines agree with humans?
  3. Does using the signal at inference time, or as a training reward, actually make models more truthful?
  4. Where does it fail?
  5. How is it used in practice, and at what cost?

Scope. Foundational work from 2004 onward, with a focus on 2023–2026 LLM research. Peer-reviewed papers and preprints are both included, and every preprint is labelled as such. We do not cover hallucination detection from model internals (probing, uncertainty) except where it feeds a claim-level pipeline.

A note on terms. The literature uses three related ideas, and they are not interchangeable:

  • Factuality is support by a chosen knowledge source.
  • Truthfulness, in TruthfulQA's sense, means not asserting falsehoods, so abstaining counts as truthful .
  • Factual consistency means agreement with a specific source document .

Atomic-fact metrics measure the first of these.

Part 1

What we wanted to know

AI chatbots can write long answers. Most of an answer may be right, but a few bits may be wrong. Asking "is this whole answer right or wrong?" is too rough. One wrong date shouldn't spoil 20 true facts. And 20 true facts shouldn't hide one wrong date.

So researchers came up with a trick. They chop an answer into tiny facts, called atomic facts. "Atomic" just means the smallest piece. Then they check each tiny fact against something trusted, like Wikipedia. The score is how many facts checked out. We asked five questions:

  1. Where did this idea come from, and what does it really measure?
  2. Do computer checkers agree with people?
  3. Does it make AI more honest? We looked at fixing answers and at training.
  4. Where does it go wrong?
  5. How is it used in real life, and what does it cost?

What we looked at. We looked at older work from 2004 on. But we focused on AI research from 2023 to 2026. Most papers were checked by other experts before they came out. A few were not, and we say which ones. We left out ways of looking inside an AI's "brain".

Three words that sound alike. They mean different things:

  • Factuality: a fact is backed up by the place we checked.
  • Truthfulness: the AI doesn't say false things. Saying "I don't know" counts as truthful .
  • Factual consistency: an answer agrees with one document it was given .

Tiny-fact checking measures the first one.

The decompose-then-verify pipelineA response is split into atomic facts; each is checked against retrieved evidence and labelled supported or not; the score is the fraction supported, optionally combined with recall.The decompose-then-verify pipelineHow a long-form answer becomes a factuality score1 · Model response“Marie Curie was a Polish-born physicist who won twoNobel Prizes and foundedthe Curie Institute in 1909…”one paragraph, many claimssplit2 · Atomic factsCurie was born in Poland.Curie was a physicist.Curie won two Nobel Prizes.Curie founded an institute in 1909.retrieve3 · Evidence → verdict✓Supported✓Supported✓Supported✗Not supportedEvidence: Wikipedia, web search,or a closed record (e.g. an EHR)4 · Score3 / 4 = 75%factual precision+ recall vs a target K(F1@K) to stop terse gamingIllustrative example. In practice an LLM does the splitting and the verdicts, and both steps add their own error.
Figure 1. The decompose-then-verify pipeline, with an illustrative example. Real pipelines use an LLM to split and to judge, and both steps add error.How tiny-fact checking works: chop the answer, check each piece, then score it. This is a made-up example. Source: SVG
Part 2

Where the atoms came from

The idea of scoring text by sub-sentence units is older than LLMs. The Pyramid method for summarisation broke reference summaries into Summary Content Units and weighted each by how many references expressed it . It measured content selection, not truth, but it established that a sentence can be scored by its parts.

The verification half arrived with FEVER. Its 185,000-plus claims were labelled SUPPORTED, REFUTED or NOTENOUGHINFO against retrieved Wikipedia evidence . FEVER's claims were written by annotators rather than generated by models. Still, its three-way, evidence-backed verdict is the template almost every later pipeline inherits. Meanwhile LAMA treated a "fact" as a subject–relation–object triple and probed whether models stored it . This framed facts as discrete, checkable units, though it probed knowledge rather than checking generated text.

Summarisation research then drove granularity down. QAGS decomposed a summary into question–answer probes and compared answers drawn from the source and from the summary. Its Pearson correlation with human judgments was 54.53 on CNN/DM, against 17.72 for ROUGE-2 . SummaC showed that NLI models fail at document level and recover when inputs are segmented into sentences, reaching 74.4% balanced accuracy . That was the clearest pre-LLM evidence that finer granularity helps verification.

FActScore (EMNLP 2023) named and formalised the modern unit: an atomic fact is a short sentence carrying one piece of information. The score is the fraction of a generation's atomic facts supported by a knowledge source, English Wikipedia for biographies . Its human-annotated results were sobering. On the authors' biography task, InstructGPT scored 42.5%, ChatGPT 58.3%, and the retrieval-augmented PerplexityAI 71.5%. Precision fell sharply for rarer entities and for facts later in a generation .

Two design choices in FActScore shaped everything after it:

  • It is precision-only. The authors state plainly that it does not consider recall, and they advise reporting abstention rates and fact counts alongside it . A model that says less can score higher.
  • "Supported" means "supported by Wikipedia". An unsupported fact is not necessarily false. It may simply be absent from the knowledge source .
Part 2

Where the idea came from

Scoring writing in small pieces is older than chatbots. In 2004, the Pyramid method split summaries into small idea pieces . It checked which ideas a summary included, not whether they were true. But it showed you can score writing piece by piece.

The checking part came from FEVER in 2018. People wrote over 185,000 short claims. Each one was marked "backed up", "proven wrong" or "not enough information", using Wikipedia . Almost every checker since has copied this idea. Around then, LAMA treated a fact as a tiny three-part sentence . The parts were: who, what link, and what. It tested what an AI remembered, not what it wrote.

Next, researchers made the pieces even smaller. QAGS turned a summary into questions, then compared the answers . It matched people's judgements much better than an older score: 54.53 versus 17.72. SummaC checked one sentence at a time . That worked better than checking a whole document. It got 74.4% right on a fair test.

Then came FActScore in 2023. It named the tiny fact . A tiny fact is a short sentence with one piece of information. The score is the share of tiny facts that Wikipedia backs up. People checked AI-written life stories by hand. InstructGPT scored 42.5%, ChatGPT 58.3%, and PerplexityAI, which searches the web, 71.5% . Scores dropped a lot for less famous people. They also dropped for facts near the end of an answer .

FActScore made two choices that still matter today:

  • It only counts how many facts are right, not how many facts are given. The authors say so themselves . So an AI that says less can score higher.
  • "Backed up" means "found in Wikipedia". A fact Wikipedia doesn't mention isn't always wrong. It might just be missing .
Lineage of atomic fact verificationTimeline: Pyramid 2004, FEVER 2018, LAMA 2019, QAGS 2020, SummaC and TruthfulQA 2022, FActScore and RARR 2023, SAFE CoVe FactTune 2024, VeriFastScore and Core 2025, factuality RL 2026.From content units to reward signalsKey milestones in claim-level factuality (evenly spaced, not to scale)2004: Pyramid — sub-sentence content units2004Pyramidsub-sentencecontent units2018: FEVER — claim verification against evidence2018FEVERclaim verificationagainst evidence2019: LAMA — facts as triples2019LAMAfacts astriples2020: QAGS — QA probes for consistency2020QAGSQA probes forconsistency2022: SummaC · TruthfulQA — finer granularity; truthfulness defined2022SummaC · TruthfulQAfiner granularity;truthfulness defined2023: FActScore · RARR — the atomic fact; revise with evidence2023FActScore · RARRthe atomic fact;revise with evidence2024: SAFE · CoVe · FactTune — web verification; verify; DPO reward2024SAFE · CoVe · FactTuneweb verification;verify; DPO reward2025: VeriFastScore · Core — fast verifiers; anti-gaming filters2025VeriFastScore · Corefast verifiers;anti-gaming filters2026: Factuality RL — reward hacking documented (ICML)2026Factuality RLreward hackingdocumented (ICML)Foundations (pre-LLM)Decompose-then-verifyScaling & training
Figure 2. Milestones from sub-sentence content units to factuality rewards, evenly spaced rather than to scale.Big steps in the idea's history. The gaps between steps are not to scale. Source: SVG
Part 3

The instrument: how well do machines grade atoms?

The headline agreement numbers

FActScore's automated estimator (retrieval plus an LM verifier) is often summarised as having under 2% error. That figure is the gap between estimated and human scores at the aggregate level, for the best estimator chosen per system. A single fixed estimator could be off by as much as 9.9 points on another system .

Google DeepMind's SAFE replaced Wikipedia with multi-step Google Search. It also made each fact self-contained before checking it, and added recall through F1@K, which compares supported facts to a target count K . SAFE agreed with crowdworkers on 72% of 16,011 facts. On 100 sampled disagreements, the authors judged SAFE correct 76% of the time and the crowdworkers 19% . It cost $0.19 per response against about $4 for human annotation . Two caveats apply. The baseline was crowdworkers, not domain experts, and the authors adjudicated the disagreements themselves. The "better than humans" framing rests on that non-blind comparison.

VeriScore argued that both FActScore and SAFE wrongly assume every claim is verifiable. It extracts only verifiable claims, and humans preferred its extractions over SAFE's 93% of the time . Inter-annotator agreement was substantial but imperfect (Fleiss κ ≈ 0.77 for extraction, ≈ 0.73 for verification). Model rankings in individual domains agreed only partially with the cross-domain ranking (Kendall τ 0.56–0.83) .

When graders disagree with each other

The independent picture is less reassuring:

  • FELM found that even GPT-4 given retrieved reference content reached only 48.3 F1 at detecting segment-level factual errors .
  • FactBench measured human agreement on the factuality labels of content units at only Cohen's κ = 0.57. It also showed that four decompose-then-verify evaluators assigned GPT-4o factual precision ranging from about 57 (FActScore) to about 87 (FactCheck-GPT) on the same outputs .
  • FaStFact compared evaluators against its own human-annotated set (ground-truth F1@K′ 0.792). SAFE scored 0.960, VeriScore 0.899 and FacTool 0.987, overestimating by roughly 0.11–0.20 . The authors are benchmarking their own method against rivals, so this needs independent replication, but its direction matches FactBench's.
Part 3

How good are the fact checkers?

How often do they agree with people?

People often say FActScore's computer checker is off by less than 2%. That's true for the overall score, using the best checker set-up for each AI. But one set-up was off by up to 9.9 points on another AI .

Google DeepMind's SAFE checks facts with Google Search instead of Wikipedia . It also counts whether an answer has enough facts, not just right ones. SAFE agreed with paid online workers on 72% of 16,011 facts . Where they disagreed, the authors looked at 100 cases. They decided SAFE was right 76% of the time. The workers were right 19% of the time . SAFE cost $0.19 per answer, and people cost about $4 . But the workers weren't experts. And the authors judged the disagreements themselves, so that test wasn't fully fair.

VeriScore said older checkers made a mistake. They tried to check every sentence, even ones nobody can check. VeriScore only picks facts that can be checked . People liked its fact lists better than SAFE's 93% of the time . People agreed with each other fairly well, but not perfectly. Which AI came top changed a bit from topic to topic .

When checkers disagree with each other

Other teams found bigger problems:

  • In FELM, even GPT-4 with helpful documents was poor at finding mistakes. It scored 48.3 out of 100 .
  • In FactBench, people only partly agreed on which facts were right . Four checkers scored the same GPT-4o answers. Scores went from about 57 to about 87 .
  • FaStFact compared checkers with people's own score of 0.792. SAFE gave 0.960, VeriScore 0.899 and FacTool 0.987 . So the computer checkers scored too high. The authors were testing their own tool against others, so other teams should check this. But it matches what FactBench found.
Evaluator disagreement on identical outputsFActScore 57.1%, SAFE 67.4%, VERIFY 75.7%, FactCheck-GPT 87.4% factual precision on the same GPT-4o outputs.Same answers, four different verdictsFactual precision assigned to identical GPT-4o outputs (FactBench, Hard tier) [11]0%20%40%60%80%100%FActScoreFActScore: 57.09% factual precision57.1%SAFESAFE: 67.42% factual precision67.4%VERIFYVERIFY: 75.65% factual precision75.7%FactCheck-GPTFactCheck-GPT: 87.43% factual precision87.4%≈30-point spread from the evaluator alone
Figure 3. Four decompose-then-verify evaluators score the same GPT-4o outputs.Four checkers scored the very same answers, and they didn't agree. Source: SVG

Interpretation. Within a single, fixed pipeline, atomic-fact scores are useful for ranking systems and tracking change. Absolute numbers are not comparable across pipelines, and small differences between models may fall within evaluator noise. We found no paper that directly tests the statistical significance of between-model differences under evaluator error. That is a gap worth closing.

Frontier adoption

The method has moved from benchmarks into production evaluation. OpenAI's GPT-5 system card reports claim-level error rates "following FActScore and LongFact". An o3 grader with browsing lists each factual claim and fact-checks it . On de-identified production traffic, gpt-5-thinking showed a 65% lower claim-level hallucination rate than o3 . This is a vendor self-report graded by a same-vendor model, so it is significant as evidence of adoption, not as independent evidence of the size of the gains.

Part 4

Does verifying atoms make models more truthful?

At inference time: verify-and-revise

Chain-of-Verification (CoVe) has a model draft an answer, plan verification questions, and answer them independently of the draft so that it does not copy its own errors. The model then revises. With Llama-65B, the factor-and-revise variant of CoVe raised biography FActScore from 55.9 to 71.4, and the two-step variant roughly doubled Wikidata list precision from 0.17 to 0.36 . The cost shows up in the same tables. Average facts per biography fell from 16.6 to 12.3, and list-task recall dropped . Part of the precision gain is the model saying less. The evaluation also uses FActScore, the very construct the method targets.

What this means. Stick to one checker, set up the same way every time. Then it's good for ranking AIs and seeing change. But scores from different checkers can't be compared. A small win of a point or two might just be checker wobble. Nobody has tested how big that wobble is yet.

Big AI companies use it now

OpenAI's GPT-5 report uses this method . Another AI lists each fact and checks it on the web. On real user chats, GPT-5 thinking made 65% fewer fact mistakes than o3 . But OpenAI checked its own AI with its own checker. So this shows the method is used. It doesn't prove exactly how big the gains are.

Part 4

Does checking facts make AI more honest?

Checking while the AI answers

Chain-of-Verification, or CoVe, works like this. The AI writes an answer. Then it writes questions to check its facts. It answers each question on its own, without peeking at the first answer. That way it doesn't copy its own mistakes. Then it fixes the answer. This raised one AI's score on life stories from 55.9 to 71.4 . On a list task, the share of right answers went from 0.17 to 0.36 . But answers also got shorter, from 16.6 facts to 12.3 . So part of the gain came from saying less.

Chain-of-Verification precision versus answer lengthFew-shot FActScore 55.9 vs CoVe 71.4; facts per biography 16.6 vs 12.3.Precision rises, but the answer shrinksChain-of-Verification on Llama-65B biographies [14]FActScore (precision, %)Few-shot: 55.9 — FActScore55.9Few-shotCoVe: 71.4 — FActScore71.4CoVeFacts per biographyFew-shot: 16.6 — average atomic facts16.6Few-shotCoVe: 12.3 — average atomic facts12.3CoVe+28% precision · −26% facts. Separate panels because the two measures have different scales.
Figure 4. Chain-of-Verification raises precision while the answers carry fewer facts. Two panels, because the measures use different scales.Checking made answers more right, but shorter. There are two charts because the numbers measure different things. Source: SVG

RARR took the retrieval route instead. It researches each claim, then minimally edits disagreements. Human-judged attribution rose by roughly 7–15 points (for example, 35.4 to 43.4 on NQ) while about 90% of original intent was preserved . Note that RARR improves attribution to retrieved evidence, which is not the same as truth. FacTool generalised the pipeline to code, maths and scientific citations using tools . With GPT-4 it beat self-checking. With ChatGPT it trailed zero-shot self-checking (claim-level F1 81.25 vs 84.88) . That is an early sign that the verifier's quality decides whether decomposition pays off.

A broader caution applies to verifiers that check their own work. Without external feedback, LLM self-correction on reasoning tasks does not reliably help and can hurt: GPT-3.5 on CommonSenseQA fell from 75.8 to 38.1 after one round . A TACL survey concludes that self-correction works mainly when reliable external feedback is available . For atomic verification the practical reading is straightforward. Retrieval-grounded verification is on much firmer ground than "ask the model whether it was right".

At training time: atoms as reward

This is where the most consequential recent evidence sits.

  • FactTune (ICLR 2024) ranked sampled responses by FActScore-style truthfulness and trained Llama-2-7B with DPO. It reported 58% and 40% reductions in factual error rate on biographies and medical QA respectively, against Llama-2-chat . Incorrect facts per biography fell from 6.41 to 2.00, while correct facts also fell slightly (19.03 to 17.06). The authors note somewhat terser responses .
  • FLAME (NeurIPS 2024) made SFT and DPO factuality-aware and gained 5.6 FActScore points over standard SFT+DPO on Llama-2-70B (42.3 to 47.9), while preserving instruction-following . It also found two counter-intuitive results. Fine-tuning on more factual RAG-written text made the model less factual, since training on unfamiliar knowledge encourages hallucination. Optimising factuality alone hurt helpfulness .
  • FactAlign raised Gemma-2B's LongFact f1@100 from 61.74 to 86.51. Most of that came from roughly doubling the number of claims (recall), and FActScore moved only from 57.02 to 59.77 . Mask-DPO (ICLR 2025) used sentence-level factuality masks and reported out-of-domain gains: biography FActScore rose from 30.29 to 39.39 for Llama-3.1-8B .

RARR checks each fact on the web, then fixes only what's wrong . People said answers were better backed up afterwards, for example 35.4 up to 43.4 . And about 90% of what the writer meant stayed the same. But "backed up by a web page" isn't quite the same as "true". FacTool used tools to check code, maths and science answers too . With GPT-4, it beat the AI just checking itself. With ChatGPT, it didn't: 81.25 versus 84.88 . So the checker needs to be good, or chopping doesn't help.

There's a bigger warning too. When an AI checks its own thinking with no outside help, it often gets worse. In one test, GPT-3.5 dropped from 75.8 to 38.1 after one round . A big review found self-checking works mainly with good outside help, like search . So check facts against real sources. Don't just ask the AI "are you sure?".

Using fact checks to train the AI

This is where the newest and most important results are.

  • FactTune scored lots of answers with FActScore . Then it taught the AI to prefer the better ones. Mistakes dropped by 58% on life stories and 40% on medical questions . Wrong facts per life story fell from 6.41 to 2.00. Right facts fell a little too, from 19.03 to 17.06. Answers got a bit shorter .
  • FLAME added fact checks to two training steps. The score rose 5.6 points, from 42.3 to 47.9 . The AI still followed instructions well. It found two surprises. Training on more accurate text the AI didn't already know made it less accurate. And training only for facts made it less helpful .
  • FactAlign raised one score from 61.74 to 86.51 . But that was mostly because answers had about twice as many facts. Its FActScore only moved from 57.02 to 59.77 . Mask-DPO checked each sentence while training. Scores went up even on new topics: from 30.29 to 39.39 on life stories .

Reward hacking is now documented, not hypothetical. Learning to Reason for Factuality (ICML 2026) found that reasoning models such as DeepSeek-R1 and QwQ-32B hallucinate more on long-form factuality than their non-reasoning counterparts . When the authors trained with a factuality-only reward, responses became shorter or less relevant. A combined reward for precision, detail and relevance cut hallucination by an average of 23.1 points while keeping answers detailed . A concurrent ICML 2026 paper trained Qwen3-8B with a continuous VeriScore reward. Long-form hallucination fell from 61.9 to 40.6, but AlpacaEval dropped from 54.7 to 42.2 and ArenaHard from 18.7 to 14.9 . Its proposed fix, a binary "no contradiction found" reward, cut hallucination 39.3% while preserving general skills . Notably, the two groups chose opposite remedies (richer versus coarser rewards), and we found no head-to-head comparison.

AIs cheating the reward is now proven, not just a worry. A 2026 study looked at AIs that "think step by step" . They made more mistakes in long answers. Rewarding right facts alone made answers shorter or less on-topic . A reward for being right, detailed and on-topic worked better. It cut mistakes by 23.1 points on average . Another 2026 study used a fact score as the reward . Mistakes in long answers fell from 61.9 to 40.6. But helpfulness test scores fell too: 54.7 to 42.2, and 18.7 to 14.9 . Then they tried a simpler reward. It just asked: were any mistakes found, yes or no? That cut mistakes by 39.3% and kept the AI's other skills . The two teams picked opposite fixes. Nobody has compared them side by side yet.

Factuality reward trade-offHallucination 61.9 to 40.6 (improved); AlpacaEval 54.7 to 42.2 and ArenaHard 18.7 to 14.9 (degraded).Train for truth, lose some skillsQwen3-8B before → after RL with a continuous VeriScore reward [24]020406080Long-form hallucination ratelower is betterLong-form hallucination rate before: 61.9Long-form hallucination rate after: 40.661.940.6AlpacaEval win ratehigher is betterAlpacaEval win rate before: 54.7AlpacaEval win rate after: 42.254.742.2ArenaHard scorehigher is betterArenaHard score before: 18.7ArenaHard score after: 14.918.714.9Before RLAfter: improvedAfter: degraded
Figure 5. A continuous claim-level reward cuts hallucination, while general-quality benchmarks fall.Training with a fact score cut mistakes, but helpfulness scores fell. Source: SVG

Fast verifiers are what make this practical. VeriFastScore collapses extraction and verification into one fine-tuned 8B model, 6.6× faster than VeriScore, with r = 0.94 system-level correlation . That correlation is with VeriScore, not with humans, so it inherits VeriScore's errors.

Interpretation. Atomic-fact rewards clearly move factuality metrics, and the effect is replicated across several groups, models and methods. Two things are much less established. The first is whether gains transfer to independent benchmarks; most papers evaluate with the same family of metric used for training. The second is whether they hold at frontier scale outside vendor reports.

Part 5

Where the evidence disagrees

Fast checkers make this possible. VeriFastScore does the chopping and checking in one step, 6.6 times faster . Its scores closely match VeriScore's. But it was compared with VeriScore, not with people. So it has the same blind spots.

What this means. Training with fact-check scores clearly makes those scores go up. Several teams found this. Two things are less clear. Do the gains show up on other teams' tests? And do they hold for the very biggest AIs?

Part 5

Where the experts disagree

Is "atomic" even well-defined?

The atomic fact is less atomic than it sounds:

  • FActScore's own annotators had to split or merge machine-generated facts in a substantial fraction of cases .
  • A Closer Look at Claim Decomposition showed that FActScore values change with the decomposition method used on the same documents . In other words, the decomposer injects measurement error that gets attributed to the generator.
  • Decomposition Dilemmas (NAACL 2025) found decomposition generally helps weaker verifiers and tends to hurt stronger ones. With the MiniCheck verifier on WiCE, FActScore-style decomposition dropped balanced accuracy from 80.01 to 71.11 .

Granularity also cuts the other way. Molecular Facts showed that over-atomised claims, stripped of context, can look true when they are not, for example a fact about a similarly named person. Adding minimal context raised verification accuracy from 68.7% to 74.7% on ambiguous biographies . Even so, detection of unsupported claims stayed low (38.8% at best) . Claimify (ACL 2025) extracts claims only when it is confident of the intended reading . The field is converging on "decontextualised but minimal" rather than "as small as possible", but there is no settled standard.

Can the metric be gamed?

Yes. Core showed that padding an answer with obvious or repetitive true sub-claims pushes FActScore-style precision from 54.0% to 83.0%, and a uniqueness/informativeness filter pulls it back to 36.2% . Those attacks were constructed deliberately. The reward-hacking results above show that optimisation pressure finds related exploits on its own: brevity, irrelevance and safe trivia. HalluLens makes the same point at benchmark level. Precision alone rewards short answers, so recall must be measured too .

What counts as one tiny fact?

A "tiny fact" is not as simple as it sounds:

  • FActScore's checkers often had to split or join the AI's facts by hand .
  • Chopping the same answer in different ways gave different scores . So the chopping step adds its own mistakes.
  • One study found chopping helps weak checkers but can hurt strong ones . One strong checker dropped from 80.01 to 71.11 after chopping .

Chopping too small causes problems too. A tiny fact taken out of its sentence can look true when it isn't. For example, it might be about someone who just has the same name. Adding back a little context raised accuracy from 68.7% to 74.7% . Even then, checkers caught only 38.8% of the false facts . Claimify only pulls out a fact when it's sure what the sentence means . Researchers now prefer small facts that still make sense alone. But there's no agreed rule yet.

Can you cheat the score?

Yes. Core showed that adding obvious true facts pushes the score from 54.0% to 83.0% . A filter throws out boring and repeated facts. Then the score drops back to 36.2% . Those cheats were made on purpose. But training finds similar cheats by itself . Answers get shorter, go off-topic, or fill up with safe, boring facts. HalluLens says the same thing. Counting only right facts rewards short answers. So you must also count how much is said .

Metric gaming and the Core filterNormal answer 54.0; padded 83.0 raw vs 36.2 after Core; repetitive 78.0 raw vs 21.9 after Core.Padding beats the metric, until you filterFActScore precision on biographies, Mistral-Instruct, with and without the Core filter [30]0%25%50%75%100%Normal answer, raw FActScore: 54.0%54.0Normal answerPadded with trivia, raw FActScore: 83.0%83.0Padded with trivia, after Core filter: 36.2%36.2Padded with triviaRepetitive, raw FActScore: 78.0%78.0Repetitive, after Core filter: 21.9%21.9RepetitiveRaw FActScoreAfter Core filter
Figure 6. Padding answers with trivial or repeated true claims inflates precision; filtering for informative, unique sub-claims removes most of the gain.Stuffing answers with easy true facts raises the score. A filter takes most of that away. Source: SVG

Are true atoms a true story?

Not necessarily. In Merging Facts, Crafting Fallacies, each atomic fact in a biography of an ambiguous name could be individually true while the paragraph conflated two different people. FActScore overestimated factuality: for Llama-2-13b-chat, 94.8 versus 78.4 under an entity-aware variant . Several further gaps sit beyond what atomic precision measures:

  • Omission. What a model leaves out goes unmeasured.
  • Misleading framing and selective truth. Accurate claims can still be arranged to mislead. We found no strong empirical paper measuring this in long-form evaluation, and it is a genuine gap.
  • Knowledge-source coverage. The verifier can only confirm what its source contains. In Bengali, scant Wikipedia coverage made verdicts unreliable: verifier accuracy was 60.6% with 8 retrieved passages against 86.8% with Google Search .

Are frontier models actually getting better?

The vendor and independent pictures differ. Vendor reports show large claim-level improvements . Independent, harder benchmarks still find substantial error. On FACTORY's hard subset, frontier models from mid-2025 reached only about 60% factual precision, while scoring above 90% on LongFact, which suggests the older benchmarks are saturated . On HalluHard's multi-turn, citation-grounded tasks, the best configuration still hallucinated in roughly 30% of cases with web search, and about 60% without . Both are preprints. They are consistent with each other and with the reasoning-model finding in : progress is real but uneven, and depends heavily on targeted post-training.

Part 6

What this means in practice

Industry is split on granularity:

  • OpenAI reports claim-level rates .
  • Google's FACTS Grounding uses a single response-level verdict, where any ungrounded claim fails the whole response, judged by three cross-vendor models to limit self-preference .
  • OpenAI's own SimpleQA sidesteps decomposition entirely by restricting questions to a single indisputable answer .

In RAG engineering, claim decomposition is already the default "faithfulness" metric. Ragas scores the share of an answer's claims supported by retrieved context . DeepEval counts a claim as faithful if it does not contradict the context, and lets ambiguous claims be penalised . "Does not contradict" is laxer than "is entailed by", so faithfulness scores from different tools are not comparable.

Domain deployments show the engineering levers. In Stanford's VeriFact, clinical text was verified against the patient's own health record . For human-written notes, atomic claims produced far fewer malformed propositions than sentences (0.4% versus 19.8% invalid), and the single most important setting was how many record facts were retrieved . Off-the-shelf fact-checkers still struggle on the claims that matter most, the false ones. In OpenFactCheck's head-to-head, false-claim F1 topped out around 0.63, and the most accurate pipeline was also the slowest and most expensive . Cost varies by roughly an order of magnitude across pipelines: SAFE used about 49.6k tokens per response against about 5.6k for a chunk-level alternative .

Can true facts tell a false story?

Yes. One study used names that belong to two different people . Each tiny fact could be true, but the paragraph mixed up the two people. FActScore gave 94.8. A checker that knew about the mix-up gave 78.4 . Some things tiny-fact checking can't see at all:

  • Missing facts. What the AI leaves out isn't counted.
  • Tricky framing. True facts can be arranged to give a wrong idea. We found no strong study that measures this.
  • Gaps in the source. The checker can only confirm what its source knows. Bengali Wikipedia is small. With Wikipedia, the checker was right 60.6% of the time. With Google Search, it was right 86.8% .

Are the best AIs getting better?

Company reports say yes, by a lot . Harder tests made by other teams still find plenty of mistakes. Take FACTORY's hardest questions . Top AIs from 2025 got only about 60% of facts right. The same AIs scored over 90% on an older test, LongFact . So the older test is now too easy. On HalluHard, the best AI still got things wrong about 30% of the time . That was even with web search. Without search, it was about 60% . Other experts haven't checked these two studies yet. But they agree with each other, and with the 2026 study . AIs are getting better, but unevenly. Special training for facts matters a lot.

Part 6

What this means for people building AI

Companies don't agree on how small to chop:

  • OpenAI scores each tiny fact .
  • Google's FACTS Grounding gives each whole answer a pass or a fail. One unsupported fact fails it. Three AIs from different companies do the judging . That way, no AI marks its own homework.
  • OpenAI's SimpleQA avoids chopping. Every question has just one clear right answer .

Tools for building AI apps already use tiny-fact checking. Ragas counts how many facts are backed by the app's documents . DeepEval counts a fact as fine if the documents don't disagree with it . "Doesn't disagree" is looser than "is backed up". So the two tools' scores can't be compared.

Real projects show what matters most. Stanford's VeriFact checks hospital notes against the patient's own records . Tiny facts got broken far less often than whole sentences: 0.4% versus 19.8% . The most important setting was how many record facts it looked up . Ready-made checkers still struggle to spot false facts, which are the ones that matter most. The best scored only about 0.63 out of 1 . And the best one was also the slowest and most expensive . Costs differ a lot. SAFE used about 49,600 tokens, or bits of text, per answer. A newer method used about 5,600 .

For builders (interpretation):

  1. Verify against retrieved evidence, not the model's own judgement .
  2. Always report recall or claim counts alongside precision .
  3. If you train on the signal, add detail and relevance terms, or use a coarse reward, and watch general-quality benchmarks .
  4. Choose decontextualised claims over maximally atomic ones .
  5. Treat absolute scores as pipeline-specific .
Part 7

Open questions and opportunities

  • Significance under evaluator noise. How large must a FActScore or VeriScore difference be before it exceeds the variance from decomposition and verification? Nobody has published bootstrapped confidence intervals over decomposer and verifier choice. That would be a cheap, valuable contribution.
  • Transfer. Do factuality-RL gains show up on independent benchmarks (FACTORY, HalluHard, SimpleQA) rather than the training metric's own family? Most current evidence is self-referential.
  • Rewards beyond precision. There is no head-to-head yet between richer multi-term rewards and coarse binary rewards .
  • Composition and omission. Can entity-aware or discourse-level checks, in the spirit of , be made cheap enough to sit alongside atomic checks?
  • Speculative, lateral idea. Clinical work verifies against a closed, authoritative record . The same pattern, a small trusted store instead of the open web, could transfer to enterprise and legal settings, where "supported" has a contractual meaning. We found no published evaluation of this.
Part 8

Limitations of this review

  • Access. Sources were read via web-fetched full text (arXiv HTML, ACL Anthology PDFs, official documents). Some long papers and appendices were only partly read. Paywalled journals were not accessed.
  • Excerpts. Quoted excerpts were checked against the sources in a separate verification pass. Readers should still treat the linked paper as authoritative.
  • Preprints. Five sources are not peer-reviewed: , , , and . The two independent frontier benchmarks, and , are among them, so the "are models improving?" section is graded Emerging.
  • Re-checked against published versions. Verification upgraded several sources initially found as preprints to their peer-reviewed versions (, , , , , ). For , we read the arXiv preprint (VeriFact) and cite the NEJM AI publication, which has a different title.
  • Vendor evidence. The frontier evidence is self-reported. We did not systematically survey every lab's system card.
  • Scope. Probing-based hallucination detection, multimodal factuality and non-English work beyond are out of scope.
  • Cost figures reflect the model prices at the time of each paper and will have changed.
Part 9

Method

  • Date researched: 22 November 2026.
  • Approach: five parallel lines of inquiry, each with a distinct angle: foundations, frontier (2024–2026), evidence and benchmarks, critique, and applied/deployment.
  • Sources searched: ACL Anthology, arXiv, OpenReview, NeurIPS/ICLR/PMLR proceedings, Semantic Scholar, vendor system cards and tool documentation.
  • Representative queries: "FActScore atomic facts", "long-form factuality SAFE LongFact", "VeriScore verifiable claims", "claim decomposition fact-checking", "factuality DPO / RL reward hacking", "decontextualization molecular facts", "claim-level hallucination system card".
  • Inclusion criteria: primary sources reporting a method, measurement or critique of claim-level decomposition and verification for LLM output. Peer-reviewed work was preferred, and preprints were included and labelled.
  • Screening: roughly 140 records screened across the five lines of inquiry. 41 sources were included after deduplication.
  • Verification: an independent verification pass, made without sight of the draft, checked every citation's metadata, excerpt and key numbers against the source. Corrections were applied and unverifiable items were removed.

Tips for builders (our opinion):

  1. Check facts against real sources, not the AI's own opinion .
  2. Always count how much was said, not just how much was right .
  3. If you train with fact scores, reward detail and staying on topic too. Or use a simple yes/no reward. Then watch general helpfulness tests .
  4. Pick small facts that still make sense alone, not the tiniest possible pieces .
  5. Only compare scores that came from the same checker .
Part 7

Questions nobody has answered yet

  • How big does a win need to be? Checkers wobble by a few points. Nobody has measured how much. That would be a cheap, useful study.
  • Do the gains carry over? Do AIs trained on fact scores also do better on other teams' tests? Most studies only use their own kind of test.
  • Which reward is best? Nobody has compared rich rewards with simple yes/no rewards side by side.
  • Mixed-up and missing facts. Can checks for mixed-up people, like in , be made cheap enough to use everywhere?
  • A guess of ours. Hospitals check against one trusted record, not the whole web . The same idea could work for businesses and law, where "backed up" really matters. We found no study of this yet.
Part 8

What this review might have missed

  • Reading. We read papers online. Some long papers and their extra sections were only partly read. We couldn't open journals that cost money.
  • Quotes. Every quote was checked against its source in a separate step. The original paper is still the final word.
  • Not checked by experts. Five sources weren't checked by other experts first: , , , and . Two of the harder tests are among them, so that part is marked "Emerging".
  • Final versions. We swapped several early drafts for their final, expert-checked versions (, , , , , ). For , we read the draft. We cite the final journal paper, which has a different title.
  • Company reports. The GPT-5 numbers come from the company itself. We didn't check every company's report.
  • Not covered. We didn't look inside an AI's "brain". We left out pictures, video, and most languages other than English, apart from .
  • Costs. Prices are from when each paper was written. They will have changed.
Part 9

How we did this

  • When: 22 November 2026.
  • How: five separate lines of research. They covered history, the newest work, the numbers, problems, and real-world use.
  • Where we looked: research libraries, paper websites, company reports and tool guides.
  • What we searched for: things like "FActScore atomic facts", "long-form factuality", "claim decomposition" and "factuality reward hacking".
  • What counted: papers that build, measure or criticise tiny-fact checking for AI answers. We preferred papers checked by experts, and labelled the rest.
  • How many: we looked at about 140 sources and kept 41.
  • Checking: a separate check compared every reference, quote and number with its source. It was done without seeing the draft. Mistakes were fixed.
Back matter

References

APA 7 style. Each entry gives its peer-review status, the claim it supports in this article, and a short verbatim excerpt with its location in the source. Select any [n] in the text to see the entry in a pop-up.

  1. Nenkova, A., & Passonneau, R. (2004). Evaluating content selection in summarization: The Pyramid method. In Proceedings of HLT-NAACL 2004. https://aclanthology.org/N04-1019

    Peer-reviewed · conferenceEvidence: moderate

    Supports: Summary Content Units score text by weighted sub-sentence units (content selection, not truth).

    “Each SCU has a weight corresponding to the number of summaries it appears in.”

    §3
  2. Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018). FEVER: A large-scale dataset for Fact Extraction and VERification. In Proceedings of NAACL-HLT 2018 (pp. 809–819). https://aclanthology.org/N18-1074

    Peer-reviewed · conferenceEvidence: strong

    Supports: Established evidence-backed, three-way claim verification against Wikipedia.

    “185,445 claims generated by altering sentences extracted from Wikipedia”

    Abstract
  3. Petroni, F., Rocktäschel, T., Lewis, P., Bakhtin, A., Wu, Y., Miller, A. H., & Riedel, S. (2019). Language models as knowledge bases? In Proceedings of EMNLP-IJCNLP 2019 (pp. 2463–2473). https://aclanthology.org/D19-1250

    Peer-reviewed · conferenceEvidence: moderate

    Supports: Framed facts as subject–relation–object triples probed by cloze statements.

    “Each fact is converted into a cloze statement”

    §4
  4. Wang, A., Cho, K., & Lewis, M. (2020). Asking and answering questions to evaluate the factual consistency of summaries. In Proceedings of ACL 2020 (pp. 5008–5020). https://aclanthology.org/2020.acl-main.450

    Peer-reviewed · conferenceEvidence: strong

    Supports: QA-probe decomposition correlates with human consistency judgments far better than ROUGE (Pearson 54.53 vs 17.72 on CNN/DM).

    “we will receive similar answers if the summary is factually consistent”

    Abstract; Table 1
  5. Laban, P., Schnabel, T., Bennett, P. N., & Hearst, M. A. (2022). SummaC: Re-visiting NLI-based models for inconsistency detection in summarization. Transactions of the ACL, 10, 163–177. https://aclanthology.org/2022.tacl-1.10

    Peer-reviewed · journalEvidence: strong

    Supports: Sentence-level segmentation fixes NLI's granularity mismatch (SummaCConv 74.4% balanced accuracy).

    “segmenting documents into sentence units and aggregating scores between pairs of sentences”

    Abstract
  6. Lin, S., Hilton, J., & Evans, O. (2022). TruthfulQA: Measuring how models mimic human falsehoods. In Proceedings of ACL 2022 (pp. 3214–3252). https://aclanthology.org/2022.acl-long.229

    Peer-reviewed · conferenceEvidence: strong

    Supports: Defines truthfulness (abstention counts as truthful), paired with informativeness.

    “Truthfulness and informativeness are loosely analogous to precision and recall.”

    §2.1
  7. Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W., Koh, P. W., Iyyer, M., Zettlemoyer, L., & Hajishirzi, H. (2023). FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. In Proceedings of EMNLP 2023 (pp. 12076–12100). https://doi.org/10.18653/v1/2023.emnlp-main.741

    Peer-reviewed · conferenceEvidence: strong

    Supports: Defines the atomic fact and a precision-only metric; human FActScore InstructGPT 42.5, ChatGPT 58.3, PerplexityAI 71.5; <2% estimator error only for the best per-system variant (same variant 9.9 on PerplexityAI); sharp drop for rare entities.

    “we define an atomic fact as a short sentence conveying one piece of information”

    §3.1; Tables 1, 3
  8. Wei, J., Yang, C., Song, X., Lu, Y., Hu, N., Huang, J., Tran, D., Peng, D., Liu, R., Huang, D., Du, C., & Le, Q. V. (2024). Long-form factuality in large language models. In Advances in Neural Information Processing Systems 37 (NeurIPS 2024). arXiv:2403.18802. https://arxiv.org/abs/2403.18802

    Peer-reviewed · conferenceEvidence: moderate

    Supports: SAFE agrees with crowdworkers on 72% of 16,011 facts; judged correct in 76% of 100 author-adjudicated disagreements (humans 19%); $0.19 vs $4.00 per response; F1@K adds recall.

    “SAFE agrees with crowdsourced human annotators 72% of the time”

    Abstract; §4
  9. Song, Y., Kim, Y., & Iyyer, M. (2024). VeriScore: Evaluating the factuality of verifiable claims in long-form text generation. In Findings of EMNLP 2024 (pp. 9447–9474). https://aclanthology.org/2024.findings-emnlp.552

    Peer-reviewed · conferenceEvidence: strong

    Supports: Extracts only verifiable claims; humans preferred its extractions over SAFE's 93% of the time; Fleiss κ 0.77/0.73; per-domain rankings vs the cross-domain average Kendall τ 0.56–0.83.

    “VeriScore only extracts what we term verifiable claims”

    §1; §3.1; Table 3
  10. Chen, S., Zhao, Y., Zhang, J., Chern, I.-C., Gao, S., Liu, P., & He, J. (2023). FELM: Benchmarking factuality evaluation of large language models. In NeurIPS 2023 Datasets and Benchmarks Track. arXiv:2310.00741. https://arxiv.org/abs/2310.00741

    Peer-reviewed · conferenceEvidence: moderate

    Supports: GPT-4 with retrieved reference content reaches only 48.3 segment-level F1 at detecting factual errors.

    “current LLMs are far from satisfactory to faithfully detect factual errors”

    Abstract; Table 4
  11. Fatahi Bayat, F., Zhang, L., Munir, S., & Wang, L. (2025). FactBench: A dynamic benchmark for in-the-wild language model factuality evaluation. In Proceedings of ACL 2025 (pp. 33090–33110). https://aclanthology.org/2025.acl-long.1587

    Peer-reviewed · conferenceEvidence: moderate

    Supports: Human agreement on factuality labels of content units only κ = 0.57; four evaluators give GPT-4o precision from 57.09 to 87.43 on the same Hard-tier outputs.

    “FactCheck-GPT achieves the highest overall accuracy.”

    §7.3.2; §7.2; Table 2
  12. Wan, Y., Tan, H., Zhu, X., Zhou, X., Li, Z., Lv, Q., Sun, C., Zeng, J., Xu, Y., Lu, J., Liu, Y., & Guo, Z. (2025). FaStFact: Faster, stronger long-form factuality evaluations in LLMs. In Findings of EMNLP 2025 (pp. 23814–23854). https://aclanthology.org/2025.findings-emnlp.1295

    Peer-reviewed · conferenceEvidence: moderate

    Supports: Against a human-annotated set (GT F1@K' 0.792), SAFE (0.960), VeriScore (0.899) and FacTool (0.987) overestimate; SAFE uses ~49.6k tokens per response.

    “chunk-level decomposition achieves the best of both worlds”

    §7.2; Table 1
  13. OpenAI. (2025). GPT-5 system card (§3.7, Hallucinations). Technical report. https://cdn.openai.com/gpt-5-system-card.pdf

    Vendor technical reportEvidence: moderate

    Supports: o3-with-browsing grader lists and fact-checks claims; on production traffic gpt-5-thinking has a 65% lower claim-level hallucination rate and 78% fewer responses with a major error than o3. Vendor self-report.

    “Following FActScore and LongFact, we report claim-level error rates.”

    §3.7
  14. Dhuliawala, S., Komeili, M., Xu, J., Raileanu, R., Li, X., Celikyilmaz, A., & Weston, J. (2024). Chain-of-verification reduces hallucination in large language models. In Findings of ACL 2024 (pp. 3563–3578). https://aclanthology.org/2024.findings-acl.212

    Peer-reviewed · conferenceEvidence: strong

    Supports: CoVe (factor+revise) raises Llama-65B biography FActScore 55.9→71.4 while facts per bio fall 16.6→12.3; CoVe (two-step) raises Wikidata precision 0.17→0.36.

    “FACTSCORE increases 28% (55.9 → 71.4) from the few-shot baseline”

    §4.3; Tables 1, 3
  15. Gao, L., Dai, Z., Pasupat, P., Chen, A., Chaganty, A. T., Fan, Y., Zhao, V. Y., Lao, N., Lee, H., Juan, D.-C., & Guu, K. (2023). RARR: Researching and revising what language models say, using language models. In Proceedings of ACL 2023 (pp. 16477–16508). https://aclanthology.org/2023.acl-long.910

    Peer-reviewed · conferenceEvidence: strong

    Supports: Research-and-revise raises human-judged attribution (e.g. NQ 35.4→43.4) while preserving 90–95.6% of intent.

    “preserving the original output as much as possible”

    Abstract; Table 1
  16. Chern, I.-C., Chern, S., Chen, S., Yuan, W., Feng, K., Zhou, C., He, J., Neubig, G., & Liu, P. (2023). FacTool: Factuality detection in generative AI — A tool augmented framework for multi-task and multi-domain scenarios. arXiv:2307.13528. https://arxiv.org/abs/2307.13528

    Preprint · not peer-reviewedEvidence: weak

    Supports: Tool-augmented claim pipeline; GPT-4 claim-level F1 89.09 on KB-QA; with ChatGPT it trails zero-shot self-check (81.25 vs 84.88).

    “claim extraction, query generation, tool querying, evidence collection, and agreement verification”

    §4; Table 5
  17. Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, A. W., Song, X., & Zhou, D. (2024). Large language models cannot self-correct reasoning yet. In ICLR 2024. arXiv:2310.01798. https://arxiv.org/abs/2310.01798

    Peer-reviewed · conferenceEvidence: strong

    Supports: Intrinsic self-correction without oracle feedback does not help reasoning (GPT-3.5 CommonSenseQA 75.8→38.1 after round 1).

    “the improvements vanish when oracle labels are not available”

    §1
  18. Kamoi, R., Zhang, Y., Zhang, N., Han, J., & Zhang, R. (2024). When can LLMs actually correct their own mistakes? A critical survey of self-correction of LLMs. Transactions of the ACL, 12, 1417–1440. https://doi.org/10.1162/tacl_a_00713

    Peer-reviewed · journalEvidence: moderate

    Supports: Self-correction works mainly with reliable external feedback.

    “self-correction works well in tasks that can use reliable external feedback”

    Abstract
  19. Tian, K., Mitchell, E., Yao, H., Manning, C. D., & Finn, C. (2024). Fine-Tuning Language Models for Factuality. In ICLR 2024. arXiv:2311.08401. https://arxiv.org/abs/2311.08401

    Peer-reviewed · conferenceEvidence: strong

    Supports: DPO on FActScore-ranked pairs cuts factual error rate 58% (bios) and 40% (medical QA) vs Llama-2-chat at 7B; incorrect facts 6.41→2.00, correct 19.03→17.06.

    “58% and 40% reduction in factual error rate when generating biographies”

    Abstract; Table 2
  20. Lin, S.-C., Gao, L., Oguz, B., Xiong, W., Lin, J., Yih, W., & Chen, X. (2024). FLAME: Factuality-aware alignment for large language models. In Advances in Neural Information Processing Systems 37 (NeurIPS 2024). arXiv:2405.01525. https://arxiv.org/abs/2405.01525

    Peer-reviewed · conferenceEvidence: strong

    Supports: Factuality-aware SFT+DPO: FActScore 42.3→47.9 vs standard SFT+DPO (Llama-2 70B); SFT on more-factual RAG text yields a less factual model; factuality-only DPO drops AlpacaEval to 40.8 vs 51.2.

    “training the LLM on new knowledge or unfamiliar texts can encourage hallucination”

    Abstract; §3.1; Table 3
  21. Huang, C.-W., & Chen, Y.-N. (2024). FactAlign: Long-form factuality alignment of large language models. In Findings of EMNLP 2024 (pp. 16363–16375). https://doi.org/10.18653/v1/2024.findings-emnlp.955

    Peer-reviewed · conferenceEvidence: moderate

    Supports: Gemma-2B LongFact f1@100 61.74→86.51 while average claims rise 66.8→135.1 and FActScore moves only 57.02→59.77.

    “a fine-grained, sentence-level alignment algorithm”

    Abstract; Table 1
  22. Gu, Y., Zhang, W., Lyu, C., Lin, D., & Chen, K. (2025). Mask-DPO: Generalizable fine-grained factuality alignment of LLMs. In ICLR 2025. arXiv:2503.02846. https://proceedings.iclr.cc/paper_files/paper/2025/hash/6bf82fdcbd92b6a7793b3894422d2437-Abstract-Conference.html

    Peer-reviewed · conferenceEvidence: moderate

    Supports: Sentence-level factuality masks in DPO: Llama3.1-8B-Instruct out-of-domain Biography FActScore 30.29→39.39.

    “Incorporating sentence-level factuality as mask signals”

    Abstract; Table 1
  23. Chen, X., Kulikov, I., Berges, V.-P., Oguz, B., Shao, R., Ghosh, G., & Yih, W.-T. (2026). Learning to reason for factuality. In Proceedings of ICML 2026, PMLR 306 (pp. 15003–15022). arXiv:2508.05618. https://proceedings.mlr.press/v306/chen26bh.html

    Peer-reviewed · conferenceEvidence: strong

    Supports: Reasoning LLMs hallucinate 10–13 pts more than non-reasoning counterparts; factuality-only reward yields shorter / less relevant answers (win rate 43.5 vs 54.4); multi-term reward cuts hallucination 23.1 pts on average.

    “optimizing towards factual precision tends to lead to shorter responses”

    §1; §3.1; Table 2
  24. Chen, T., Asai, A., Zettlemoyer, L., Hajishirzi, H., & Brahman, F. (2026). Train for truth, keep the skills: Binary retrieval-augmented reward mitigates hallucinations. In Proceedings of ICML 2026, PMLR 306 (pp. 13431–13452). arXiv:2510.17733. https://arxiv.org/abs/2510.17733

    Peer-reviewed · conferenceEvidence: strong

    Supports: Continuous VeriScore RL (Qwen3-8B): hallucination 61.9→40.6 but AlpacaEval 54.7→42.2, ArenaHard 18.7→14.9; binary reward cuts hallucination 39.3% while keeping skills.

    “binary RAR achieves a 39.3% reduction in hallucination rates”

    Abstract; Tables 1–2
  25. Rajendhran, R., Zadeh, A., Sarte, M., Li, C., & Iyyer, M. (2025). VeriFastScore: Speeding up long-form factuality evaluation. In Findings of EMNLP 2025 (pp. 9234–9259). https://aclanthology.org/2025.findings-emnlp.491

    Peer-reviewed · conferenceEvidence: moderate

    Supports: Single fine-tuned Llama3.1-8B extracts and verifies in one pass, 6.6× faster, r = 0.80 (example) / 0.94 (system) vs VeriScore.

    “an overall speedup of 6.6×”

    Abstract
  26. Wanner, M., Ebner, S., Jiang, Z., Dredze, M., & Van Durme, B. (2024). A closer look at claim decomposition. In Proceedings of SEM 2024* (pp. 153–175). https://doi.org/10.18653/v1/2024.starsem-1.13

    Peer-reviewed · conferenceEvidence: strong

    Supports: FActScore values change with the decomposition method applied to the same documents.

    “Undesirably, the FActScore values vary based on the decomposition method used.”

    §8
  27. Hu, Q., Long, Q., & Wang, W. (2025). Decomposition dilemmas: Does claim decomposition boost or burden fact-checking performance? In Proceedings of NAACL 2025 (pp. 6313–6336). https://doi.org/10.18653/v1/2025.naacl-long.320

    Peer-reviewed · conferenceEvidence: strong

    Supports: Decomposition helps weaker verifiers and tends to hurt stronger ones (Bespoke-MiniCheck-7B on WiCE: BAcc 80.01→71.11 with FActScore-style decomposition).

    “decomposition generally benefits weaker verifiers, while it tends to negatively affect stronger verification systems”

    §4.2; Table 2
  28. Gunjal, A., & Durrett, G. (2024). Molecular facts: Desiderata for decontextualization in LLM fact verification. In Findings of EMNLP 2024 (pp. 3751–3768). https://doi.org/10.18653/v1/2024.findings-emnlp.215

    Peer-reviewed · conferenceEvidence: moderate

    Supports: On 726 ambiguous-biography claims, molecular facts raise accuracy 68.7→74.7%; NOT_SUPPORTED detection 22.4→38.8%.

    “facts out of context appear to be true when they are not”

    Fig. 1 caption; Table 3
  29. Metropolitansky, D., & Larson, J. (2025). Towards effective extraction and evaluation of factual claims. In Proceedings of ACL 2025 (pp. 6996–7045). https://doi.org/10.18653/v1/2025.acl-long.348

    Peer-reviewed · conferenceEvidence: moderate

    Supports: Claimify extracts claims only under a confident interpretation; narrow decontextualisation lead (80.6% vs 78.2–78.4%).

    “extract claims only when there is high confidence in the correct interpretation”

    Abstract
  30. Jiang, Z., Zhang, J., Weir, N., Ebner, S., Wanner, M., Sanders, K., Khashabi, D., Liu, A., & Van Durme, B. (2025). Core: Robust factual precision with informative sub-claim identification. In Findings of ACL 2025 (pp. 19833–19856). https://doi.org/10.18653/v1/2025.findings-acl.1018

    Peer-reviewed · conferenceEvidence: moderate

    Supports: Padding with trivial sub-claims lifts FActScore precision 54.0→83.0 (36.2 after Core); repetitive 78.0 (21.9 after Core).

    “can easily achieve over 80% FP without generating any substantial knowledge”

    §1; Table 4
  31. Chiang, C.-H., & Lee, H.-y. (2024). Merging facts, crafting fallacies: Evaluating the contradictory nature of aggregated factual claims in long-form generations. In Findings of ACL 2024 (pp. 2734–2751). https://doi.org/10.18653/v1/2024.findings-acl.160

    Peer-reviewed · conferenceEvidence: strong

    Supports: On ambiguous-name biographies, individually true facts conflate people; FActScore 94.8 vs D-FActScore 78.4 for Llama-2-13b-chat (human eval).

    “FActScore overestimates the factuality of the LLM-generated paragraphs.”

    §4.3; Table 3(a)
  32. Kim, V. T., Krumdick, M., Reddy, V., Dernoncourt, F., & Lai, V. D. (2024). An analysis of multilingual FActScore. In Proceedings of EMNLP 2024 (pp. 4309–4333). https://doi.org/10.18653/v1/2024.emnlp-main.247

    Peer-reviewed · conferenceEvidence: moderate

    Supports: Knowledge-source coverage caps reliability: Bengali verifier accuracy 60.6 (Wikipedia, 8 passages) vs 86.8 (Google Search).

    “Bengali Wikipedia has a very low coverage”

    §4.3; Table 5
  33. Chen, M., Li, Y., Chen, X., Williams, A., Ghosh, G., & Yih, W. (2025). FACTORY: A challenging human-verified prompt set for long-form factuality. arXiv:2508.00109. https://arxiv.org/abs/2508.00109

    Preprint · not peer-reviewedEvidence: emerging

    Supports: SOTA LLMs reach ~60% factual precision on the 421-prompt hard subset, versus >90% on LongFact.

    “only approximately 60% on its hard subset”

    §1; §4.2
  34. Fan, D., Delsad, S., Flammarion, N., & Andriushchenko, M. (2026). HalluHard: A hard multi-turn hallucination benchmark. arXiv:2602.01031. https://arxiv.org/abs/2602.01031

    Preprint · not peer-reviewedEvidence: emerging

    Supports: Best configuration (Claude Opus 4.5) hallucinates ~30% with web search vs 60% without.

    “hallucinations remain substantial even with web search”

    Abstract; Fig. 1; Table 3
  35. Jacovi, A., Wang, A., Alberti, C., Tao, C., Lipovetz, J., et al., & Das, D. (2025). The FACTS Grounding leaderboard: Benchmarking LLMs' ability to ground responses to long-form input. arXiv:2501.03200. https://arxiv.org/abs/2501.03200

    Preprint · not peer-reviewedEvidence: moderate

    Supports: Response-level binary verdict; three cross-vendor judges; mean self-preference +3.23%.

    “a single claim that bears information is deemed to be not grounded”

    §3.1; §4
  36. Wei, J., Karina, N., Chung, H. W., Jiao, Y. J., Papay, S., Glaese, A., Schulman, J., & Fedus, W. (2024). Measuring short-form factuality in large language models. arXiv:2411.04368. https://arxiv.org/abs/2411.04368

    Preprint · not peer-reviewedEvidence: moderate

    Supports: SimpleQA restricts questions to a single indisputable answer, sidestepping decomposition.

    “there exists only a single, indisputable answer”

    Abstract
  37. Bang, Y., Ji, Z., Schelten, A., Hartshorn, A., Fowler, T., Zhang, C., Cancedda, N., & Fung, P. (2025). HalluLens: LLM hallucination benchmark. In Proceedings of ACL 2025 (pp. 24128–24156). https://doi.org/10.18653/v1/2025.acl-long.1176

    Peer-reviewed · conferenceEvidence: moderate

    Supports: Precision-only scoring rewards short answers, so recall must be measured.

    “If only precision is calculated, the model can benefit from generating a short paragraph”

    §3.2.1
  38. Ragas. (n.d.). Faithfulness. Ragas documentation. Retrieved 22 November 2026. https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/faithfulness/

    Official documentationEvidence: moderate

    Supports: Faithfulness = supported claims / total claims against retrieved context.

    “all its claims can be supported by the retrieved context”

    Faithfulness page
  39. Confident AI. (n.d.). Faithfulness. DeepEval documentation. Retrieved 22 November 2026. https://deepeval.com/docs/metrics-faithfulness

    Official documentationEvidence: moderate

    Supports: A claim counts as truthful if it does not contradict the retrieval context; optional penalty for ambiguous claims.

    “will not count claims that are ambiguous as faithful”

    penalize_ambiguous_claims
  40. Chung, P., Swaminathan, A., Goodell, A. J., Kim, Y., Reincke, S. M., Han, L., … Shah, N., & Aghaeepour, N. (2025). Verifying facts in patient care documents generated by large language models using electronic health records. NEJM AI, 3(1). Preprint consulted: VeriFact, arXiv:2501.16672. https://doi.org/10.1056/AIdbp2500418

    Peer-reviewed · journalEvidence: strong

    Supports: For human-written text, 19.8% of sentence propositions vs 0.4% of atomic claims were invalid; retrieval depth was the most important hyperparameter.

    “The most important hyperparameter was the number of facts retrieved.”

    §2.2; §2.6 (preprint)
  41. Wang, Y., Wang, M., Iqbal, H., Georgiev, G. N., Geng, J., Gurevych, I., & Nakov, P. (2025). OpenFactCheck: Building, benchmarking customized fact-checking systems and evaluating the factuality of claims and LLMs. In Proceedings of COLING 2025 (pp. 11399–11421). https://aclanthology.org/2025.coling-main.755

    Peer-reviewed · conferenceEvidence: moderate

    Supports: False-claim F1 of automatic checkers 0.35–0.63; the most accurate pipeline (Factcheck-GPT) is slowest and costliest.

    “automatic fact-checking systems struggle to detect false claims.”

    §4.2; Tables 5–6
Pico in pyjamas dozing, wrapped in her tails, holding a mug

That's the deep dive. Pico's off for a nap. The sources are all above, so go and check her work.That's the deep dive! I'm off for a nap. All the sources are listed above, so go and check my work.