What Does Verification Mean for Legal AI?
Legal AI can retrieve, summarize, and analyze documents. Verification requires something more explicit: defined checks, evidentiary support, coverage, and repeatable results.
AI systems can churn through piles of legal documents and review them at a scale that would have been impossible only a few years ago. They can retrieve relevant provisions, classify documents, summarize long agreements, identify possible issues, and produce answers with accurate citations to source material. While these capabilities are useful, they do not all solve the same problem. Furthermore, there are additional challenges that are today left unsolved. One of the more difficult tasks legal teams face is verifing that various details are correct and that these details and their premises entail the resulting conclusions and stated claims.
Verification is a very different process than simply citing a source. For example, a legal review system can identify a handful of issues in a transaction and provide supporting citations for each. The citations tell the reviewer something about what the system found but it does not necessarily establish coverage; what questions were evaluated, what exact context was reviewed, what evidence was required for each determination, if every relevant condition was considered, or what happened when the available evidence was insufficient.
This is important because a review result and a verified determination have different semantics. A finding is an output, whereas verification requires an account of how that output was established. Existing standards provide a beneficial starting point. ISO/IEC 17029 defines verification as:[1]
“the confirmation of a claim, through the provision of objective evidence, that specified requirements have been fulfilled”
NIST’s glossary definition is very similar.[2] These definitions do capture two essential elements of verification: a specified claim or requirement and objective evidence against which it can be evaluated. For legal verification, however, two additional concepts need to be made much more explicit:
-
Negative and unresolved outcomes. Verification should not be limited to confirming that a requirement has been fulfilled. The available evidence may instead contradict the claim, or it may be insufficient to resolve it.
-
Coverage. A verification result should differentiate an evaluated-and-satisfied requirement from one that was never evaluated. An absence of findings does not, by itself, establish that every relevant question was checked.
For legal AI, we therefore propose a broader more applicable working definition:
Verification is the evaluation of specified claims or requirements against objective evidence to determine what is supported, contradicted, unresolved, and actually evaluated.
A compact model is:
Each component addresses a different source of uncertainty in legal analysis.
Facts: establishing what the evidence supports
Legal conclusions often depend on facts distributed across multiple sources: agreements, schedules, amendments, disclosure materials, correspondence, corporate records, and documents in a virtual data room. Retrieving a document is not the same as establishing a fact from that document. In formal logic, an argument is sound when its premises are true and its inference (entailment) is valid. Verification therefore has to address both sides of the problem: if the factual premises are actually supported by the evidence and if the conclusion follows from those premises. But what does it mean for factual premises to be supported?
Suppose an agreement requires a particular asset to be transferred at closing. A search system may locate several documents that mention the asset, identify the operative provision, and return passages that appear relevant. None of those operations, by themselves, establish that the required transfer actually occurred. Verification therefore requires us to recognize the difference between relevant evidence and evidence sufficient to establish a proposition.
This becomes important whenever the legal question depends on more than the presence of language in a document. The relevant evidence may establish the required fact, establish an incompatible fact, or fail to resolve the question at all. A verification system needs to preserve those possibilities rather than treating document retrieval as proof. For this reason, the factual part of verification is concerned with propositions such as:
- the closing occurred on a specified date;
- a required document was executed;
- a particular party delivered the document;
- an amendment changed the operative obligation;
- a representation applies to the relevant entity or period.
The output is not just a collection of passages; it is an evidentiary basis for determining which facts can presently be established. If the facts are not established, then the ensuing arguments will not be sound.
Logic: determining what follows from the established facts
Even when the underlying facts are established and available, a second problem remains. The system must determine how those facts relate to the requirement being evaluated. In formal logic, this is the question of validity: if the conclusion follows from the premises. Consider a simplified closing condition:
If the transaction closes, the seller must deliver Document X at closing.
Assume the evidence establishes that the transaction closed. Whether the requirement is satisfied still depends on whether the evidence also establishes delivery of Document X.
A citation to the closing provision does not answer that question. A citation to a document resembling Document X may not answer it either. The determination depends on the relationship among the contractual requirement, the relevant conditions, and the facts established from the evidence.
This relationship can become substantially more complex in actual agreements. Legal provisions frequently contain qualifications, exceptions (sometimes deeply nested exceptions-to-exceptions), thresholds, defined terms, temporal conditions, cross-references, dependencies on schedules, and references to other obligations. An analysis may therefore require several factual predicates before a conclusion is supported.
Logical errors can happen even when the relevant legal rule and underlying facts have been correctly identified. Some of these errors have familiar forms in formal logic. Courts have occasionally identified them explicitly when a legal argument depends on an invalid inference from a rule or factual condition.
One example is denying the antecedent. The invalid form is:
If P, then Q.
Not P.
Therefore, not Q.
In State ex rel. Banks v. Drummond, the Washington Supreme Court considered a statute governing county contracts for professional services. The relevant statutory condition could be represented approximately as:
If the required procedures are not followed,
then the contract is unlawful.
The county argued, in effect:
The required procedures were followed.
Therefore, the contract is lawful.
The court identified the inference as the fallacy of denying the antecedent. Compliance with the specified procedures eliminated one basis for concluding that the contract was unlawful, but it did not establish that the contract was lawful. The county still needed an independent source of authority to enter the contract.[3]
A related error appears when reasoning moves in the opposite direction. Affirming the consequent has the form:
If P, then Q.
Q.
Therefore, P.
In In re Stewart Foods, Inc., the Fourth Circuit considered whether obligations under a non-executory retirement agreement gave rise to a claim in bankruptcy. The district court reasoned in part that because rejection of an executory contract can give rise to a general unsecured claim, treating the retirement obligation as a claim effectively meant treating the contract as rejected. The relevant inference can be simplified as:
If a debtor rejects a contract,
then a general unsecured claim exists.
A general unsecured claim exists.
Therefore, the debtor must have rejected the contract.
The Fourth Circuit expressly identified this as affirming the consequent and held that the conclusion did not follow from the premises. The existence of a claim did not establish that the contract had been rejected; a monetary obligation under a non-executory contract could itself give rise to a claim under the Bankruptcy Code. The court reversed and remanded the district court’s decision.[4]
These examples show why identifying the applicable language and underlying facts is not enough. The relationship between them also has to be valid. A source can be correctly retrieved, a fact can be correctly extracted, and the resulting conclusion can still fail because the inference itself does not follow. This is one reason verification benefits from making the dependency structure of a determination explicit enough to inspect. If the evidence establishes both closing and delivery, the requirement may be supported. If the evidence establishes that the required document was not delivered, the requirement may be contradicted. If closing is established but delivery cannot be determined from the available materials, the result is unresolved. For a concrete transaction example showing how supported premises, unresolved branches, coverage, and contractual qualifications affect the resulting determination, see the Howco–Drone USA case study.[5]
The purpose is not to eliminate interpretation from legal work. Interpretation determines, among other things, what a provision means and which factual conditions need to be considered. Verification instead makes it possible to see which interpretation was applied, which facts were relied upon, and how those facts produced the reported result.
Coverage: a clean result versus a check that never ran
Coverage is a separate problem from correctness. Suppose a system reviews an acquisition agreement and returns seven findings. A reviewer cannot infer from those seven findings that every other relevant requirement was examined. The absence of a finding may mean that a check was performed and no issue was identified. It may also mean that the issue was never evaluated. Those states are operationally different.
This is particularly important in diligence workflows because the relevant question is often not simply whether the system found something interesting. The reviewer may need to know whether a defined set of representations, closing conditions, requested documents, or other requirements was actually examined.
A large context window does not by itself establish analytical coverage. The ability to place a document within a model’s context is different from demonstrating that every relevant part of that context was effectively considered. Research on long-context language models has repeatedly found this to be important in practice. Liu et al. showed that performance can vary substantially depending on where relevant information appears within a long input, with models often performing worse when the relevant material occurs in the middle of the context.[6] RULER subsequently showed that strong performance on simple retrieval tests does not necessarily extend to tasks involving multiple relevant facts, aggregation, or multi-step relationships, and that performance can decline as both context length and task complexity increase.[7]
Retrieval-based approaches introduce a related boundary. An answer may be well supported by the passages selected for analysis without establishing that every relevant part of the underlying record was selected or examined. Coverage is therefore a separate property from the quality of an individual finding. A reviewer may need to know not only whether a reported conclusion is correct, but whether each defined requirement was actually evaluated.
A verification system therefore needs a representation of coverage in addition to findings. For each defined check, a good system should clearly separate the following review states:
- evaluated and supported;
- evaluated and contradicted;
- evaluated but unresolved;
- not evaluated.
This turns silence into something interpretable. Instead of asking whether the system happened to flag an issue, the reviewer can ask whether a particular requirement was checked and inspect the resulting determination. This becomes more important as the number of documents and potential checks grows. A list of findings can describe what was surfaced. Coverage describes the analytical space that was actually examined.
Repeatability: applying the same check under changing evidence
Legal review is not static. Data rooms are often incomplete; new documents arrive, schedules are updated, agreements are amended, and factual assumptions change during the course of a transaction.
When a useful check has been defined explicitly, it can be applied again as the evidence changes. Consider a diligence review where a representation initially cannot be verified because the relevant schedule is missing. The first result may be unresolved. If the schedule is later added to the data room, the same check can be executed against the expanded evidence set.
Repeatability is facilitative because the analytical question does not have to be reconstructed from scratch each time the underlying materials change. The same requirement, factual dependencies, and decision criteria can be applied to the new state of the transaction. This does not make the check infallible. Its assumptions may be incomplete. The encoded interpretation may be wrong. A relevant exception may have been omitted. The evidence extraction itself may need correction.
The advantage is that these elements are inspectable. If the interpretation or rule changes, the revised check can be applied consistently to the same evidence and to subsequent versions of the transaction record. Repeatability therefore supports another important property: auditability. A reviewer can examine not only the latest answer but also the basis on which the answer was produced.
Verification requires an explicit treatment of uncertainty
Legal analysis frequently operates under incomplete evidence. An effitive verification system should represent that condition directly. For a proposition under review, three outcomes are particularly beneficial:
Supported. The required conditions are established by the available evidence.
Contradicted. The available evidence establishes facts incompatible with the proposition or requirement being tested.
Unresolved. The available evidence establishes neither the proposition nor its contradiction with sufficient support.
The unresolved state is not a failure of the system. In many diligence workflows, it may be one of the most important results. It identifies the boundary between what the available record establishes and what still requires additional evidence, interpretation, or human investigation. Without such a state, systems are pressured toward binary conclusions even when the evidence does not justify them. Verification should do the opposite: it should preserve uncertainty when uncertainty is present in the underlying record.
Verification and legal judgment solve different problems
Verification does not replace legal judgment, and it does not make AI-assisted review unnecessary. Human reviewers remain responsible for questions that depend on professional judgment, materiality, market-standardization, negotiation context, legal interpretation, and decisions about which risks are prioritized in a particular transaction. AI systems can be used for reading, retrieving, classifying, summarizing, and analyzing large bodies of legal language.
Verification addresses a different problem: whether defined propositions or requirements can be evaluated against an evidentiary record in a way that makes the reasoning, coverage, and result inspectable.
The relationship is similar to automated analysis in software engineering. An automated check does not replace the engineer who designed the system or reviews the code. It makes a particular class of question explicit and applies the corresponding test consistently. The usefulness of the check depends on whether the question can be defined precisely enough to evaluate.
Legal verification has the same boundary. Not every legal question should be reduced to an automated check. The approach is most promising when the requirement can be stated explicitly, the relevant evidence can be identified, and the cost of missing, misreading, or failing to evaluate that requirement is significant.
What verification adds
Retrieval answers where relevant information may be found. Extraction identifies information contained in that material. Analysis can organize, summarize, and reason about it. Verification adds another question:
Given a defined requirement and the available evidence, what can actually be established, and was the requirement evaluated at all?
That question requires more than a generated conclusion. It requires an explicit relationship among evidence, facts, analytical conditions, and results. A working definition is therefore:
Verification is the evaluation of specified claims or requirements against objective evidence to determine what is supported, contradicted, unresolved, and actually evaluated.
Or, more compactly:
Facts + Logic + Coverage + Repeatability
The important demarcation that we are calling out isn’t between human review and automated review, or between conventional software and AI, but rather between a system that reports what it found and a system that can account for what it checked.
References
[1] International Organization for Standardization, ISO/IEC 17029:2019, Conformity assessment — General principles and requirements for validation and verification bodies, §3.3 (“verification”). ↩
[2] National Institute of Standards and Technology, “Verification,” Computer Security Resource Center Glossary. ↩
[3] State ex rel. Banks v. Drummond, 187 Wash. 2d 157, 385 P.3d 769 (Wash. 2016), as amended Feb. 8, 2017. ↩
[4] In re Stewart Foods, Inc., 64 F.3d 141 (4th Cir. 1995). ↩
[5] Good Terms, “Case Study: Verifying R&Ws Against the Transaction Record” (Oct. 11, 2026). ↩
[6] Nelson F. Liu et al., “Lost in the Middle: How Language Models Use Long Contexts,” Transactions of the Association for Computational Linguistics 12 (2024): 157–173. DOI 10.1162/tacl_a_00638. ACL Anthology ↩
[7] Cheng-Ping Hsieh et al., “RULER: What’s the Real Context Size of Your Long-Context Language Models?” arXiv:2404.06654(2024). ↩
Disclaimer: This article is for informational purposes only and is not legal advice. For advice about a specific matter, consult qualified legal counsel.