A Citation Is Not Verification
A citation can be accurate and still fail to establish a conclusion. Verification asks whether the evidence is sufficient and whether the conclusion survives the relevant evidentiary record.
More accurate citations are one of the clearest improvements in legal AI over the last couple years. Besides reduced AI hallucinations in general, simple methods of deterministically verififying citation details to mitigate mistakes have had real practical benefits. This more trivial example of verification however is not what this article is about.
The act of referencing documents helps both humans and AI ground the reasoning. Instead of producing an unsupported answer, a system can identify the agreement, provision, schedule, case, or other source material on which its response is based. This gives the reviewer a path back to the underlying record and makes errors easier to spot. While that traceability is valuable, it doesn’t by itself establish that the cited evidence is sufficient to support the conclusion. This entailment problem as well as the issue of coverage is what we will focus on here.
In Concept 001, we defined verification as the evaluation of specified claims or requirements against objective evidence, with explicit treatment of what is supported, contradicted, unresolved, and actually evaluated.[1] That definition separates several properties that are easy to collapse in an answer-and-citation interface.
A citation primarily establishes provenance:
Citation: Where did this information come from?
Verification has to establish something further:
Verification: Does the available evidence establish the conclusion?
This difference is still meaningful even when the cited source is real, relevant, and accurately quoted.
A correct source can still support the wrong conclusion
A recent Court of Federal Claims decision provides an unusually clear example. In Scale AI, Inc. v. United States, Scale challenged the National Geospatial-Intelligence Agency’s treatment of a competing proposal in a procurement for data-labeling services.[2] The competitor had initially failed to include certain software-license terms required by the solicitation. Scale argued that the omission required NGA to eliminate the competitor from consideration.
Scale relied on language in the solicitation stating that NGA would strictly enforce proposal submission requirements. The language existed and was plainly relevant to the dispute. The problem was what came next.
The same provision later stated that the government may consider a nonconforming proposal noncompliant and remove the offeror from consideration. It also expressly allowed the contracting officer to seek clarification of certain minor or clerical errors. The court concluded that Scale’s presentation of the provision was “materially incomplete” and that the selective quotation was misleading. The provision did not require the automatic elimination Scale claimed.
This is an illustrative verification failure because the source itself was not fabricated. The quoted language was real and it concerned precisely the issue being litigated. Correcting the citation location did not cure the deeper problem because the proposition being asserted was not established by the relevant provision read as a whole. The evidentiary relationship looked something like this:
“Strictly enforce proposal requirements.”The existential witness: one location in one source supports the claim.
NGA may treat a nonconforming proposal as noncompliant.
Certain errors may be clarified.
- Provenance✓
- Relevance✓
- Sufficiency×
A citation can therefore succeed as provenance without supplying sufficient inferential warrant for the conclusion.
A citation is an existential witness
Formal logic provides an insightful way to describe what a citation establishes. Let P be a proposition and E the relevant evidentiary record. A supporting citation can establish:
(There exists some evidence e in E such that e supports P.)
The citation acts as an existential witness. It demonstrates that supporting evidence exists. But the existence of supporting evidence does not establish that the proposition survives the rest of the relevant record. There may also be evidence that acts as a defeater, i.e., contradicts, qualifies, supersedes, or otherwise defeats the proposed conclusion:
(There exists some evidence e in E such that e defeats P.)
Claiming that no such evidence exists is much stronger:
(There does not exist any evidence e in E such that e defeats P.)
By quantifier negation, this is equivalent to:
(For every piece of evidence e in E, e does not defeat P.)
This exposes a basic asymmetry between citation and verification. Finding one supporting source requires only an existential result. Establishing that a conclusion survives the relevant evidentiary record requires accounting for evidence that could change that conclusion. Coverage therefore becomes part of the logical problem. Suppose a system evaluates only a retrieved subset E' of the relevant record E. It may be able to establish:
(There does not exist any evidence e in E’ such that e defeats P.)
In other words, no contradictory evidence was found in the material examined. But that does not establish:
(There does not exist any evidence e in E such that e defeats P.)
The first statement concerns the material that was examined. The second concerns the relevant evidentiary record. That difference is the difference between not finding a problem and establishing that the relevant space was checked for the problem. This is why several excellent citations can coexist with an unsupported conclusion. Each citation may provide genuine local evidence for P, while contrary evidence remains elsewhere in the record or outside the portion of the record actually evaluated.
A citation can establish:
(There exists some evidence e in E such that e supports P.)
But a citation does not establish that the search for defeating evidence was exhaustive. Verification has to ask the more difficult question:
(Does there exist some evidence e in E such that e defeats P?)
And before the absence of such evidence can carry much weight, the verification process must also account for what portion of E was actually evaluated. The Howco–Drone USA case study provides a detailed transaction example: evidence can strongly support an individual proposition while the broader contractual conclusion remains unresolved because other branches or portions of the relevant evidentiary domain have not been established.[3] This is why verification must account for coverage: which portion of the relevant evidentiary record was actually evaluated.
Context can change what the cited words establish
Sometimes the missing information is not in another document. It can be a few words away. In Sharei Torah v. Hendel, a New York court considered a statement saying, in substance, that if a federal court opened a civil docket, the plaintiff would promptly move to remand.[4] Hendel later characterized the statement as an admission that the plaintiff would move to remand, omitting the condition concerning the opening of the federal docket.
Every important word in the shortened quotation appeared in the original. The court nevertheless found the truncation deceptive because it removed the condition precedent. The logical structures were different:
Original:
IF federal docket opens
THEN plaintiff will move to remand
became:
Plaintiff will move to remand
The citation could lead a reviewer to genuine text in a genuine source and still misrepresent what that source established. The Supreme Court made a related observation in Masson v. New Yorker Magazine: “an exact quotation out of context can distort meaning.”[5] Literal correspondence between quoted words and source text is therefore not sufficient to establish faithful analytical use of the source.
The same problem appears in legal AI whenever a retrieved passage is evaluated independently of language that controls its scope. A sentence can be authentic, relevant, and locally supportive while definitions, exceptions, conditions, amendments, or surrounding provisions change the result.
Provenance, relevance, and sufficiency are different properties
These examples separate three properties that an answer-and-citation interface can easily collapse: provenance, relevance, and sufficiency.
Provenance asks whether the cited material actually comes from the identified source.
Relevance asks whether that material bears on the proposition being evaluated.
Sufficiency asks whether the available evidence carries the evidentiary burden required by the conclusion.
A source can perform well on the first two and still fail on the third. A source may be probative without being dispositive: it can provide genuine evidence bearing on a proposition without supplying enough evidence to establish the proposition. Suppose a system concludes:
The company owns all intellectual property
developed by its employees.
and cites an executed invention-assignment agreement. The citation may be impeccable; it identifies a real agreement containing assignment language. Yet the conclusion may require several additional facts: whether the agreement covers the relevant rights, whether it applies to the correct contributor, whether every relevant contributor executed an effective assignment, whether later agreements modified the result, and whether countervailing evidence exists elsewhere in the record. If the proposition requires:
A AND B AND C
then excellent citations for A and B do not establish the proposition if C remains unresolved, illustrating that citation density is a poor substitute for evidentiary sufficiency. Ten locally correct references can still leave a required condition unestablished.
Citation research already separates some of these questions
Research on LLM citations has begun to formalize related distinctions. Gao, Yen, Yu, and Chen’s ALCE benchmark evaluates generated answers along separate dimensions of correctness and citation quality.[6] Citation quality is itself divided into citation recall, which asks whether generated statements are supported by cited passages, and citation precision, which evaluates whether individual citations are relevant to the claims that they accompany. On the ELI5 benchmark used in the study, even the best systems in their experiments lacked complete citation support about half the time. The demarcation is useful: an answer can be correct while poorly cited, or well cited while incomplete or incorrect. Treating citation presence as a proxy for correctness would erase these differences.
A recent NIST research demonstrator goes further by evaluating citations along three separate dimensions: faithfulness, completeness, and sufficiency.[7] This is particularly relevant. A citation can faithfully reproduce what a source says while still omitting material context. Even a source can be faithfully and completely represented while still carrying too little evidentiary weight for the conclusion being asserted.
These properties concern the relationship among a claim, a source, and the broader evidentiary record. The presence of a citation is only the beginning of that analysis.
Local support is not global support
Most citation interfaces are naturally local. A passage is retrieved because it is related to a question, an answer is generated from that passage, and the passage is attached to the answer. Legal determinations are frequently nonlocal.
A provision may be qualified by a definition elsewhere in the agreement. An amendment may supersede the original language. A disclosure schedule may create an exception. A side letter may change one party’s rights. A document elsewhere in the data room may establish a fact incompatible with the apparent conclusion. The question is whether the conclusion survives consideration of the evidentiary context. A cited passage can support the answer in isolation without resolving that question.
Formal analysis of citation also exposes a coverage problem: a citation tells the reviewer what evidence was used, but it does not, by itself, establish what evidence was considered. These are different statements:
"This passage supports the conclusion."
"This conclusion remains supported after
the relevant evidence has been evaluated."
The second claim is substantially stronger. A reviewer who sees several convincing citations may still be unable to determine whether an unmentioned condition was satisfied, contradicted, unresolved, or never evaluated. As the evidentiary record grows, that missing coverage information becomes increasingly consequential.
Citations remain part of verification
None of this diminishes the value of citations. A verification process without provenance would create another serious problem. Reviewers need to be able to inspect the evidence supporting factual premises and trace determinations back to the underlying record.
Citations therefore belong inside a verification record. Their role is to connect propositions and factual findings to source material. They do not replace the other parts of the analysis.
A verification result must still account for the proposition being evaluated, the facts required to establish it, evidence that supports or contradicts those facts, unresolved conditions, and whether the relevant questions were actually evaluated. Concept 001 describes this broader structure as:
Verification = Facts + Logic + Coverage + Repeatability
Citations are essential to the Facts side of that structure because they preserve provenance and make evidence inspectable but the existence of a citation does not establish valid inference, evidentiary sufficiency, or analytical coverage. A citation can tell us where supporting evidence came from whereas verification asks whether the conclusion remains warranted after the relevant evidence has been considered.
References
[1] Good Terms, What Does Verification Mean for Legal AI?, Concept 001. ↩
[2] Scale AI, Inc. v. United States, No. 26-155 (Fed. Cl. Aug. 18, 2026), at 20–21. ↩
[3] Good Terms, Case Study: Verifying R&Ws Against the Transaction Record, Case Study. ↩
[4] Sharei Torah v. Hendel, 2026 NY Slip Op 50994(U), 89 Misc. 3d 1218(A) (Sup. Ct. Queens County June 1, 2026). ↩
[5] Masson v. New Yorker Magazine, Inc., 501 U.S. 496, 515 (1991). ↩
[6] Tianyu Gao, Howard Yen, Jiatong Yu & Danqi Chen, Enabling Large Language Models to Generate Text with Citations, Proceedings of EMNLP 2023, 6465–6488. ↩
[7] National Institute of Standards and Technology, Agentic Research Evaluation Probes, citation faithfulness, completeness, and sufficiency evaluation demonstrator. ↩
Disclaimer: This article is for informational purposes only and is not legal advice. For advice about a specific matter, consult qualified legal counsel.