Within legal scholarship and practice, among the most pervasive tasks is the interpretation of texts. And within legal interpretation, perhaps the most pervasive inquiry is the search for “ordinary meaning.” Jurists often treat ordinary meaning analysis as an empirical inquiry, aiming to discover a fact about how people understand language. When evaluating ordinary meaning, interpreters rely on dictionary definitions or patterns of common usage, increasingly via “legal corpus linguistics” approaches. However, the most central question about these popular methods remains open: Do they reliably reflect ordinary meaning? This Article presents experiments that assess whether (a) dictionary definitions and (b) common usage data reflect (c) how people actually understand language today.
The Article elaborates the implications of two main experimental results. First, neither the dictionary nor legal corpus linguistics methods reliably track ordinary people’s judgments about meaning. This finding shifts the argumentative burden to jurists who rely on these tools to identify “ordinary meaning” or “original public meaning”: these views must articulate and demonstrate a reliable method of analysis. Moreover, this divergence illuminates several interpretive fallacies. For example, advocates of legal corpus linguistics often contend that the nonappearance of a specific use in a corpus indicates that the use is not part of the relevant term’s ordinary meaning. The experiments reveal this claim to be a “Nonappearance Fallacy.” Ordinary meaning exceeds datasets of common usage — even very large ones.
Second, dictionary and legal corpus linguistics verdicts diverge dramatically from each other. Part of that divergence is explained by the finding that broad dictionary definitions tend to direct interpreters to extensive interpretations, while data of common usage tends to point interpreters to more prototypical cases. This divergence suggests two different criteria that are often relevant in interpretation: a more extensive criterion and a more narrow criterion. Although dictionaries and legal corpus linguistics might, in some cases, help us identify these criteria, a hard legal-philosophical question remains: Which of these two criteria should guide the interpretation of terms and phrases in legal texts? Insofar as there is no compelling case to prefer one, the results suggest that dictionary definitions, legal corpus linguistics, or even other more scientific measures of meaning may not be equipped in principle to deliver simple and unequivocal answers to inquiries about the so-called “ordinary meaning” of legal texts.
Introduction
Within legal scholarship and practice, among the most pervasive tasks is the interpretation of texts. And within legal interpretation, among the most pervasive inquiries is the search for ordinary meaning. Across the interpretation of contracts,
wills,
trusts,
deeds,
patents,
statutes,
regulations,
treaties,
and constitutions,
legal theorists and practitioners regularly evaluate the text’s ordinary meaning.
This Article focuses primarily on interpretation of American contracts and statutes, but ordinary meaning is also of global legal significance: “[E]very legal system recognizes the importance of ordinary meaning . . . .”
This is for good reason: “[W]hat method of . . . interpretation would view the ordinary meaning of words as completely irrelevant?”
Even legal theorists who advocate looking beyond ordinary meaning acknowledge that, in interpretation, “one certainly begins there.”
Despite this general agreement concerning ordinary meaning’s legal relevance, there is significant debate about how exactly to elaborate the concept of ordinary meaning. On one common view, the ordinary meaning of a text is what its words would communicate to ordinary people. In some circumstances, legal theories seek the original ordinary meaning or “original public meaning” of a text: what its words would have communicated to people at some past time, such as the time a contract or will is formalized, a bill becomes a statute, or a constitution or treaty is ratified.
Various legal debates concern this original or historical ordinary meaning of a text, especially in statutory and constitutional contexts,
but also in contractual ones.
Analysis of ordinary meaning is extraordinarily common — and increasingly so. The ubiquity of ordinary meaning analysis can be explained in part by the ubiquity of legal interpretation,
where ordinary meaning is “the most fundamental principle.”
Analysis of the ordinary, plain, or natural meaning underlies popular approaches to contract interpretation
in the United States and abroad,
which remains the “most important source of commercial litigation.”
Ordinary meaning analysis also informs interpretation of patents,
trusts, and wills.
An empirical study of the 2005–2008 Supreme Court Terms found that the majority of Supreme Court Justices “referenced text/plain meaning and Supreme Court precedent more frequently than any of the other interpretive tools.”
In light of the three most recent Supreme Court appointments, ordinary meaning analysis will likely continue to hold a significant place.
Consider Justice Kavanaugh’s view of interpreting statutes: “Under the ‘best reading’ inquiry, the question is only how the words [of a statute] would be read by an ordinary user of the English language. That’s why textualists rely on dictionaries.”
Similarly, Justice Gorsuch frequently assesses the ordinary meaning of legal texts. In his first Supreme Court opinion, he analyzed a statute’s “ordinary meaning,” citing both (i) the Oxford English Dictionary as well as (ii) common patterns of language use.26
Ordinary meaning analysis is sometimes associated with conservative legal thought. But the consideration of ordinary meaning is broader. In Muscarello v. United States,27 Justice Breyer analyzed the statute’s “ordinary” meaning, employing similar methods to those utilized by Justice Gorsuch.
Justice Breyer referred to (i) dictionary definitions
and (ii) patterns of common word usage in “computerized newspaper databases.”
Similarly, although ordinary meaning analysis is often associated with textualism and formalism,
a diverse range of theories endorse the relevance of ordinary meaning as one criterion in legal interpretation.
And on other views, ordinary meaning is crucial evidence of other criteria: when interpreting contracts by aiming to uncover and preserve the parties’ mutual intent, “the words of an integrated agreement remain the most important evidence of intention.”
Of course, emphasis on ordinary meaning is not central to all theories of interpretation, and notable detractors question the empirical assumptions required to discover ordinary meaning: Do judges actually have the ability, insight, or tools to determine the ordinary meaning of legal texts?
That critique highlights a crucial insight. Ordinary meaning inquiries are often understood as empirical ones, which aim to discover descriptive facts about meaning.
Theories holding that a legal text must be applied consistently with its ordinary meaning do not typically characterize their project as a normative inquiry. Rather than debating how a text should be understood by some ideal person, these theories ask how a text would in fact be understood by ordinary people.
There are several empirical methods commonly used to inquire into a text’s ordinary meaning, including consulting dictionary definitions or using “legal corpus linguistics” to analyze patterns of language usage across a corpus.
The popularity of these methods is not difficult to explain. Dictionary use and the dominant form of legal corpus linguistics are both relatively easy to employ. Moreover, they often seem objective, neutral, and scientific.
Both methods are also increasingly popular. The Supreme Court cites dictionaries more today than ever before.
Legal corpus linguistics is certainly less prevalent, but it has also grown in use and esteem.
The Supreme Court has examined patterns of word use through newspaper databases,40 and state supreme courts have searched corpora including the Corpus of Contemporary American English (COCA).
The growing use of dictionaries and legal corpus linguistics is likely to continue.
Yet, despite the enthusiasm surrounding dictionaries and legal corpus linguistics, there is surprisingly little work assessing what these tools actually do in legal interpretation. Although the use of dictionaries and legal corpus linguistics seems to grow more sophisticated,
their reliability has never been rigorously assessed.
There are important critiques of these methods from external theoretical perspectives,
but we might also take an internal perspective, considering whether these methods succeed on their own terms. Theories relying on these tools typically assume that dictionaries and legal corpus linguistics reliably reflect ordinary meaning,
what “the ordinary user of the English language might understand,”
but the question remains: Is this assumption true?
This Article develops a novel method to test dictionaries and legal corpus linguistics. This method provides evidence about the reliability of these tools and, by extension, of the theories that rely on them. Insofar as a legal interpretive theory relies upon dictionary definitions or patterns of word usage, the study here also provides evidence about the success of such legal theories. Part I outlines the background to these debates and legal theories in which ordinary meaning analysis is significant. Part II surveys the tools that provide interpretive evidence for those theories, including the consultation of dictionary definitions and patterns of linguistic usage across corpora.
Parts III and IV consider the accuracy and reliability of these putative measures of ordinary meaning. One reason that these tools have not yet been tested is that such a test may seem impossible, particularly when we are considering original meaning in some historical time period. How can we evaluate the accuracy of an eighteenth-century dictionary or even a corpus linguistics search from the 1980s without facts about the way in which the contested term was actually understood at the time (that is, without the best data about its original ordinary meaning)? However, what we can do is evaluate whether modern uses of contemporary dictionaries and legal corpus linguistics reflect terms’ modern meanings. If a tool (for example, dictionary use) performs poorly in modern interpretation, so long as there are no historically distinguishing factors (for example, reasons that use of an eighteenth-century dictionary would be more reliable in historical interpretation than use of a modern dictionary is in modern interpretation), this fact gives us some evidence that the method is also unreliable in estimating original meaning.
To answer this central question — do dictionaries and legal corpus linguistics reliably reflect ordinary meaning? — the Article presents a robust experimental investigation of dictionaries and legal corpus linguistics. Experimental studies of ordinary people, law students, and United States judges all indicate that the way people understand ordinary terms and phrases (for example, “vehicle” or “carrying a firearm”) varies systematically from what a dictionary definition or relevant legal corpus linguistics’ usage data would indicate about the meaning.
After presenting and analyzing a range of experiments, the Article identifies ten noteworthy results and elaborates on their implications. Judges, law students, and ordinary people were strikingly similar in their ordinary conceptual judgments, use of dictionaries, and use of legal corpus linguistics. But within each group there existed considerable disagreement. For example, subjects within each group were divided on whether a canoe is a vehicle.
Legal corpus linguistics users tended to identify narrow, prototypical examples (for example, a car is a vehicle) rather than nonprototypical examples (for example, an airplane is a vehicle), while dictionary users tended in the opposite direction, supporting more extensive judgments (for example, a pair of roller skates is a vehicle). As such, the verdicts of legal corpus linguistics and dictionary use often diverged from each other.
Moreover, both legal corpus linguistics and dictionary use diverged from ordinary meaning — at a conservative divergence rate between 20–35%. That is, across all the levels of expertise — ordinary people, law students, and judges — the data suggest that relying on just a dictionary definition or selection of legal corpus linguistics data would lead users to the wrong judgment about “ordinary meaning” fairly often, once in every three to five cases. And that conservative rate may not tell the whole story. The experiments included a number of relatively easy categorizations, such as whether a car is a vehicle. Insofar as real legal decisions often concern comparatively harder categorizations, such as whether an airplane is a vehicle, it is also instructive to consider the maximum error rate: For example, what percentage of judges using dictionary or legal corpus linguistics methods evaluated the hardest interpretive question incorrectly? Across all levels of expertise, relying on a dictionary definition or corpus linguistics data led 80–100% of users to the incorrect verdict.
For theories committed to the notion of a single “ordinary meaning” that determines legal outcomes across a range of cases and contexts, the results suggest that dictionaries and legal corpus linguistics — two central tools of discovering ordinary meaning — are unreliable in interpretation. This shifts the argumentative burden to those who rely on these tools to provide both a principled defense of their use and an empirical demonstration of how error can be avoided. Without such an account and demonstration, it is hard to take seriously claims that these tools should be used by judges to determine “ordinary meaning” or that these tools deliver objective, restrained, or nondiscretionary interpretive outcomes.
Moreover, the results help identify common fallacies in the use of dictionaries and legal corpus linguistics. For instance, consider what this Article calls the “Nonappearance Fallacy” — namely, the (false) claim that absence of a usage from a large corpus indicates that the usage is not part of the ordinary meaning. One might be surprised to learn that in some modern corpora, there are no airplanes referred to as “vehicles.” Although it is tempting to conclude from this that airplanes are not included in the ordinary meaning of “vehicles,” the experiments show this to be too quick: the majority of ordinary people, law students, and judges evaluated airplanes to be vehicles. The broader insight underlying these critiques is that ordinary meaning diverges from ordinary use. Although courts and commentators sometimes conflate these concepts, there is a crucial distinction between ordinary meaning and what is typically spoken and recorded (for example, in a corpus).
For theories uncommitted to, or even skeptical of, the notion of a single “ordinary meaning,” the results illuminate two different criteria that are often relevant in assessments of the meaning of legal texts: a more extensive criterion and a more narrow, prototypical criterion. For example, an extensive criterion indicates that airplanes, canoes, and even drones are vehicles, while a more prototypical criterion indicates that these entities are not vehicles (only cars and similar entities are vehicles). The findings show that dictionaries or legal corpus linguistics sometimes help clarify one of these criteria — often dictionaries point to the extensive criterion and legal corpus linguistics to the prototypical one — but a hard legal-philosophical question remains: Which of these should serve as a criterion in legal interpretation? Good reasons underlie both criteria, especially the fact that ordinary people may understand the relevant concepts with both. As such, the results suggest that dictionary definitions, legal corpus linguistics, or even other more scientific measures of meaning may not be equipped in principle to deliver simple and unequivocal answers to inquiries about the ordinary meaning of law. Instead, in hard cases, legal interpreters will likely have to look beyond the dictionary and corpus — to the legal text’s context, history, and purpose.
* Assistant Professor of Law, Georgetown University Law Center; Research Affiliate, ETH Zurich Center for Law & Economics. Many thanks to the editors of the Harvard Law Review; to audiences at the Association for American Law Schools, Experimental Methods in Legal Scholarship, American Law and Economics Association, and Law & Corpus Linguistics at BYU conferences; and for thoughtful comments from Ellen Aprill, Elliott Ash, Stefan Bechtold, Clark Cunningham, Jesse Egbert, Richard Elliott, William Eskridge, Ed Finnegan, Tammy Gales, Stephen Galoob, Tobias Gesche, Abbe Gluck, Neal Goldfarb, Stefan Gries, Ivar Hannikainen, David Hoffman, Daniel Keller, Josh Kleinfeld, Markus Kneer, Josh Knobe, Issa Kohler-Hausmann, Anita Krishnakumar, Thomas Lee, Tom Lyon, Jamie Macleod, Stefan Magen, Daniel Markovits, John Mikhail, Stephen Mouritsen, Victoria Nourse, Karolina Prochownik, Kevin Reuter, Martin Schonger, Scott Shapiro, Dan Simon, Brian Slocum, Larry Solan, Roseanna Sommers, Alexander Stremitzer, Mila Versteeg, Pascale Willemsen, Gideon Yaffe, and Evan Zoldan.