Within legal scholarship and practice, among the most pervasive tasks is the interpretation of texts. And within legal interpretation, perhaps the most pervasive inquiry is the search for “ordinary meaning.” Jurists often treat ordinary meaning analysis as an empirical inquiry, aiming to discover a fact about how people understand language. When evaluating ordinary meaning, interpreters rely on dictionary definitions or patterns of common usage, increasingly via “legal corpus linguistics” approaches. However, the most central question about these popular methods remains open: Do they reliably reflect ordinary meaning? This Article presents experiments that assess whether (a) dictionary definitions and (b) common usage data reflect (c) how people actually understand language today.
The Article elaborates the implications of two main experimental results. First, neither the dictionary nor legal corpus linguistics methods reliably track ordinary people’s judgments about meaning. This finding shifts the argumentative burden to jurists who rely on these tools to identify “ordinary meaning” or “original public meaning”: these views must articulate and demonstrate a reliable method of analysis. Moreover, this divergence illuminates several interpretive fallacies. For example, advocates of legal corpus linguistics often contend that the nonappearance of a specific use in a corpus indicates that the use is not part of the relevant term’s ordinary meaning. The experiments reveal this claim to be a “Nonappearance Fallacy.” Ordinary meaning exceeds datasets of common usage — even very large ones.
Second, dictionary and legal corpus linguistics verdicts diverge dramatically from each other. Part of that divergence is explained by the finding that broad dictionary definitions tend to direct interpreters to extensive interpretations, while data of common usage tends to point interpreters to more prototypical cases. This divergence suggests two different criteria that are often relevant in interpretation: a more extensive criterion and a more narrow criterion. Although dictionaries and legal corpus linguistics might, in some cases, help us identify these criteria, a hard legal-philosophical question remains: Which of these two criteria should guide the interpretation of terms and phrases in legal texts? Insofar as there is no compelling case to prefer one, the results suggest that dictionary definitions, legal corpus linguistics, or even other more scientific measures of meaning may not be equipped in principle to deliver simple and unequivocal answers to inquiries about the so-called “ordinary meaning” of legal texts.
Within legal scholarship and practice, among the most pervasive tasks is the interpretation of texts. And within legal interpretation, among the most pervasive inquiries is the search for ordinary meaning. Across the interpretation of contracts,1×1. See, e.g., Cal. Civ. Code § 1644 (West 2018) (“The words of a contract are to be understood in their ordinary and popular sense . . . .”); Jowett, Inc. v. United States, 234 F.3d 1365, 1369 (Fed. Cir. 2000) (“We give the words of the agreement their ordinary meaning unless the parties mutually intended and agreed to an alternative meaning.” (quoting Harris v. Dep’t of Veterans Affs., 142 F.3d 1463, 1467 (Fed. Cir. 1988))). wills,2×2. See, e.g., Cal. Prob. Code § 21122 (West 2018) (“The words of an instrument are to be given their ordinary and grammatical meaning unless the intention to use them in another sense is clear and their intended meaning can be ascertained.”). trusts,3×3. See, e.g., id. deeds,4×4. See, e.g., Lambert v. Pritchett, 284 S.W.2d 90, 90 (Ky. 1955) (“Terms are to be construed and understood according to their plain, ordinary, and popular sense . . . .”); Burdette v. Bruen, 191 S.E. 360, 363 (W. Va. 1937). patents,5×5. See, e.g., Phillips v. AWH Corp., 415 F.3d 1303, 1321 (Fed. Cir. 2005) (“[T]he ‘ordinary meaning’ of a claim term is its meaning to the ordinary artisan . . . .”). statutes,6×6. See, e.g., Moskal v. United States, 498 U.S. 103, 108 (1990) (“‘In determining the scope of a statute, we look first to its language,’ giving the ‘words used’ their ‘ordinary meaning.’” (citations omitted) (first quoting United States v. Turkette, 452 U.S. 576, 580 (1981); and then twice quoting Richards v. United States, 369 U.S. 1, 9 (1962))). regulations,7×7. See, e.g., Bowles v. Seminole Rock & Sand Co., 325 U.S. 410, 414 (1945) (noting that the Court’s only tools “are the plain words of the regulation and any relevant interpretations of the Administrator”). See generally Christensen v. Harris County, 529 U.S. 576 (2000). treaties,8×8. See, e.g., Vienna Convention on the Law of Treaties art. 31, opened for signature May 23, 1969, 1155 U.N.T.S. 331, 340 (“A treaty shall be interpreted in good faith in accordance with the ordinary meaning to be given to the terms of the treaty in their context and in light of its object and purpose.”). See generally Curtis J. Mahoney, Treaties as Contracts: Textualism, Contract Theory, and the Interpretation of Treaties, 116 Yale L.J. 824 (2007); Brian G. Slocum & Jarrod Wong, The Vienna Convention and the Ordinary Meaning of International Law, 46 Yale J. Int’l L. (forthcoming 2021). and constitutions,9×9. See generally, e.g., Lawrence B. Solum, The Constraint Principle: Original Meaning and Constitutional Practice (Apr. 3, 2019) (unpublished manuscript) (on file with the Harvard Law School Library). legal theorists and practitioners regularly evaluate the text’s ordinary meaning.
This Article focuses primarily on interpretation of American contracts and statutes, but ordinary meaning is also of global legal significance: “[E]very legal system recognizes the importance of ordinary meaning . . . .”10×10. Daniel A. Farber, The Hermeneutic Tourist: Statutory Interpretation in Comparative Perspective, 81 Cornell L. Rev. 513, 516 (1996) (reviewing Interpreting Statutes: A Comparative Study (D. Neil MacCormick & Robert S. Summers eds., 1991)) (discussing a study that examined the relevance of ordinary meaning to legal interpretation in jurisdictions including Argentina, Britain, Finland, France, Germany, Italy, Poland, Sweden, and the United States). This is for good reason: “[W]hat method of . . . interpretation would view the ordinary meaning of words as completely irrelevant?”11×11. Id. (citing Edward Rubin, Book Review, 41 Am. J. Compar. L. 128, 139 (1993) (reviewing Interpreting Statutes, supra note 11)). Even legal theorists who advocate looking beyond ordinary meaning acknowledge that, in interpretation, “one certainly begins there.”12×12. Felix Frankfurter, Some Reflections on the Reading of Statutes, 47 Colum. L. Rev. 527, 535 (1947).
Despite this general agreement concerning ordinary meaning’s legal relevance, there is significant debate about how exactly to elaborate the concept of ordinary meaning. On one common view, the ordinary meaning of a text is what its words would communicate to ordinary people. In some circumstances, legal theories seek the original ordinary meaning or “original public meaning” of a text: what its words would have communicated to people at some past time, such as the time a contract or will is formalized, a bill becomes a statute, or a constitution or treaty is ratified.13×13. See Thomas R. Lee & Stephen C. Mouritsen, Judging Ordinary Meaning, 127 Yale L.J. 788, 826 (2018). Various legal debates concern this original or historical ordinary meaning of a text, especially in statutory and constitutional contexts,14×14. Debate is especially divisive in the statutory and constitutional context. See generally, e.g., Victoria Nourse, Reclaiming the Constitutional Text from Originalism: The Case of Executive Power, 106 Calif. L. Rev. 1 (2018); Solum, supra note 10. Following Justice Lee and Professor Stephen Mouritsen, I use “original public meaning” to refer to a legal text’s communicative content (or “ordinary meaning”) at the relevant time. See Lee & Mouritsen, supra note 14, at 825–26. For example, the original public meaning of a 1967 statute is that text’s ordinary meaning in 1967. but also in contractual ones.15×15. See, e.g., Stephen C. Mouritsen, Contract Interpretation with Corpus Linguistics, 94 Wash. L. Rev. 1337 (2019).
Analysis of ordinary meaning is extraordinarily common — and increasingly so. The ubiquity of ordinary meaning analysis can be explained in part by the ubiquity of legal interpretation,16×16. See generally, e.g., Aharon Barak, Purposive Interpretation in Law (2005); William N. Eskridge, Jr., Interpreting Law: A Primer on How to Read Statutes and the Constitution (2016); Robert A. Katzmann, Judging Statutes (2014); Antonin Scalia & Bryan A. Garner, Reading Law: The Interpretation of Legal Texts (2012); Slocum, supra note 1; Lawrence M. Solan, The Language of Judges (1993); Lawrence M. Solan, The Language of Statutes: Laws and Their Interpretation (2010); Richard H. Fallon, Jr., The Meaning of Legal “Meaning” and Its Implications for Theories of Legal Interpretation, 82 U. Chi. L. Rev. 1235 (2015). where ordinary meaning is “the most fundamental principle.”17×17. Slocum, supra note 1. Analysis of the ordinary, plain, or natural meaning underlies popular approaches to contract interpretation18×18. See generally, e.g., Shawn Bayern, Contract Meta-interpretation, 49 U.C. Davis L. Rev. 1097, 1099 (2016); Ronald J. Gilson et al., Contract and Innovation: The Limited Role of Generalist Courts in the Evolution of Novel Contractual Forms, 88 N.Y.U. L. Rev. 170, 171 n.1 (2013) (“Textualist interpretation . . . looks to a contract’s formal language . . . .”); Robert E. Scott, The Death of Contract Law, 54 U. Toronto L.J. 369 (2004); Gregory Klass, Contract Exposition and Formalism (2017) (unpublished manuscript) (on file with the Harvard Law School Library). in the United States and abroad,19×19. See, e.g., Arnold v. Britton  UKSC 36 (appeal taken from EWCA (Civ.)) (“[T]he reliance placed in some cases on commercial common sense and surrounding circumstances . . . should not be invoked to undervalue the importance of the language of the provision which is to be construed.”). which remains the “most important source of commercial litigation.”20×20. Ronald J. Gilson, Charles F. Sabel & Robert E. Scott, Text and Context: Contract Interpretation as Contract Design, 100 Cornell L. Rev. 23, 25 (2014). Ordinary meaning analysis also informs interpretation of patents,21×21. See generally, e.g., Tun-Jen Chiang & Lawrence B. Solum, The Interpretation-Construction Distinction in Patent Law, 123 Yale L.J. 530 (2013). trusts, and wills.22×22. E.g., Cal. Prob. Code § 21122 (West 2018); Naomi R. Cahn, Alyssa A. DiRusso & Susan N. Gary, Wills, Trusts, and Estates in Focus 144 (2019).
An empirical study of the 2005–2008 Supreme Court Terms found that the majority of Supreme Court Justices “referenced text/plain meaning and Supreme Court precedent more frequently than any of the other interpretive tools.”23×23. Anita S. Krishnakumar, Statutory Interpretation in the Roberts Court’s First Era: An Empirical and Doctrinal Analysis, 62 Hastings L.J. 221, 251 (2010). In light of the three most recent Supreme Court appointments, ordinary meaning analysis will likely continue to hold a significant place.24×24. See, e.g., Brett M. Kavanaugh, Fixing Statutory Interpretation, 129 Harv. L. Rev. 2118, 2125 (2016) (reviewing Robert A. Katzmann, Judging Statutes (2014)). Consider Justice Kavanaugh’s view of interpreting statutes: “Under the ‘best reading’ inquiry, the question is only how the words [of a statute] would be read by an ordinary user of the English language. That’s why textualists rely on dictionaries.”25×25. Id. at 2150 n.158 (“Dictionaries may not provide authoritative, binding interpretations of the language of a statute, but they do tell courts something about how the ordinary user of the English language might understand that statutory language.”). Similarly, Justice Gorsuch frequently assesses the ordinary meaning of legal texts. In his first Supreme Court opinion, he analyzed a statute’s “ordinary meaning,” citing both (i) the Oxford English Dictionary as well as (ii) common patterns of language use.26×26. Henson v. Santander Consumer USA Inc., 137 S. Ct. 1718, 1722–24 (2017).
Ordinary meaning analysis is sometimes associated with conservative legal thought. But the consideration of ordinary meaning is broader. In Muscarello v. United States,27×27. 524 U.S. 125 (1998). Justice Breyer analyzed the statute’s “ordinary” meaning, employing similar methods to those utilized by Justice Gorsuch.28×28. Id. at 127–28. Justice Breyer referred to (i) dictionary definitions29×29. Id. at 128–29. and (ii) patterns of common word usage in “computerized newspaper databases.”30×30. Id. at 129.
Similarly, although ordinary meaning analysis is often associated with textualism and formalism,31×31. See Victoria Nourse, Textualism 3.0: Statutory Interpretation After Justice Scalia, 70 Ala. L. Rev. 667, 681 (2019). See generally Klass, supra note 19 (discussing interpretive formalism in contract law). a diverse range of theories endorse the relevance of ordinary meaning as one criterion in legal interpretation.32×32. See, e.g., Eskridge, supra note 17, at 35 (“There are excellent reasons for the primacy of the ordinary meaning rule.”); Lee & Mouritsen, supra note 14, at 788 (claiming that “most everyone agrees” that the ordinary meaning rule predominates in legal interpretation); Lawrence B. Solum, Surprising Originalism: The Regula Lecture, 9 ConLawNOW 235, 251–59 (2018); see also Harvard Law School, The Scalia Lecture: A Dialogue with Justice Kagan on the Reading of Statutes, YouTube, at 8:29 (Nov. 25, 2015), https://youtu.be/dpEtszFT0Tg [http://perma.cc/3BCF-FEFR] (“We’re all textualists now.”). And on other views, ordinary meaning is crucial evidence of other criteria: when interpreting contracts by aiming to uncover and preserve the parties’ mutual intent, “the words of an integrated agreement remain the most important evidence of intention.”33×33. Restatement (Second) of Contracts § 212 cmt. b (Am. L. Inst. 1981). And in determining the purpose of a statute, purposivists often ask, “what would a reasonable human being intend this specific language to accomplish?” Stephen Breyer, Lecture, On the Uses of Legislative History in Interpreting Statutes, 65 S. Cal. L. Rev. 845, 854 (1992) (internal quotations omitted). See generally Henry M. Hart, Jr. & Albert M. Sacks, The Legal Process: Basic Problems (1995).
Of course, emphasis on ordinary meaning is not central to all theories of interpretation, and notable detractors question the empirical assumptions required to discover ordinary meaning: Do judges actually have the ability, insight, or tools to determine the ordinary meaning of legal texts?34×34. See, e.g., Fallon, supra note 17, at 1235; Cass R. Sunstein, There is Nothing that Interpretation Just Is, 30 Const. Comment. 193, 194–95 (2015).
That critique highlights a crucial insight. Ordinary meaning inquiries are often understood as empirical ones, which aim to discover descriptive facts about meaning.35×35. See, e.g., Randy E. Barnett, Interpretation and Construction, 34 Harv. J.L. & Pub. Pol’y 65, 66 (2011) (“It cannot be overstressed that the activity of determining semantic meaning at the time of enactment required by the first proposition is empirical, not normative.” (citing Keith E. Whittington, Constitutional Interpretation: Textual Meaning, Original Intent, and Judicial Review 6 (1999))). Theories holding that a legal text must be applied consistently with its ordinary meaning do not typically characterize their project as a normative inquiry. Rather than debating how a text should be understood by some ideal person, these theories ask how a text would in fact be understood by ordinary people.
There are several empirical methods commonly used to inquire into a text’s ordinary meaning, including consulting dictionary definitions or using “legal corpus linguistics” to analyze patterns of language usage across a corpus.36×36. There is a growing literature on legal corpus linguistics. This Article is primarily concerned with what is currently the most popular and prevalent form of legal corpus linguistics, which seeks to identify “ordinary meaning” via evidence of common usage (for example, data about the frequency of word usage). This approach is exemplified by Justice Lee and Professor Mouritsen’s work. See Lee & Mouritsen, supra note 14. This is also the form of corpus linguistics that has been used or cited thus far in actual legal decisions. See, e.g., Carpenter v. United States, 138 S. Ct. 2206, 2238–39 (2018) (Thomas, J., dissenting) (citing corpus linguistic evidence that “[t]he phrase ‘expectation(s) of privacy’ does not appear in” Founding-era sources, id. at 2238); Caesars Ent. Corp. v. Int’l Union of Operating Eng’rs, 932 F.3d 91, 95 (3d Cir. 2019) (citing “most common synonyms” of the relevant term, “previously,” and the “words that most often co-occurred” with it); Wilson v. Safelite Grp., Inc., 930 F.3d 429, 444 (6th Cir. 2019) (Thapar, J., concurring in part and concurring in the judgment) (taking frequency data as evidence of ordinary meaning); State v. Lantis, 447 P.3d 875, 880–81 (Idaho 2019) (noting that in the corpus linguistics search concerning the phrase “disturbing the peace,” “88.4% referenced a public, external, physical peace,” id. at 881, and that this finding supported the court’s “conclusion that ‘disturbing the peace’ has a meaning that nearly always refers to public, external peace,” id.); Richards v. Cox, 450 P.3d 1074, 1079 (Utah 2019) (citing frequency data to interpret the meaning of “employment”); Fire Ins. Exch. v. Oltmanns, 416 P.3d 1148, 1163 n.9 (Utah 2018) (Durham, J., concurring in part and concurring in the result) (advocating corpus linguistics as a tool to identify the “most frequent meaning” and “most common meaning” (internal quotations omitted)); State v. Rasabout, 356 P.3d 1258, 1275–82 (Utah 2015) (Lee, A.C.J., concurring in part and concurring in the judgment) (taking frequency data as evidence of ordinary meaning); In re Adoption of Baby E.Z., 266 P.3d 702, 725–26 (Utah 2011) (Lee, J., concurring in part and concurring in the judgment) (same).
This Article refers to this particular form of corpus linguistics as “legal corpus linguistics.” It is, of course, crucial to acknowledge the possibility that some new method of legal corpus linguistics might be developed. Moreover, such a method might, perhaps, be shown to escape the problems raised here. The Article’s burden-shifting conclusion, see infra section VI.C, pp. 798–801, invites precisely this response. The Article’s critiques should therefore not be seen as critiques of the broader linguistic field of “corpus linguistics,” which contributes to many other questions beyond legal debates about “ordinary meaning” and “original public meaning.” The popularity of these methods is not difficult to explain. Dictionary use and the dominant form of legal corpus linguistics are both relatively easy to employ. Moreover, they often seem objective, neutral, and scientific.37×37. See, e.g., Lawrence M. Solan, Can Corpus Linguistics Help Make Originalism Scientific?, 126 Yale L.J.F. 57 (2016).
Both methods are also increasingly popular. The Supreme Court cites dictionaries more today than ever before.38×38. See John Calhoun, Note, Measuring the Fortress: Explaining Trends in Supreme Court and Circuit Court Dictionary Use, 124 Yale L.J. 484, 497, 497 fig.1, 502 fig.3 (2014). Legal corpus linguistics is certainly less prevalent, but it has also grown in use and esteem.39×39. See generally Lee & Mouritsen, supra note 14; Lee J. Strang, How Big Data Can Increase Originalism’s Methodological Rigor: Using Corpus Linguistics to Reveal Original Language Conventions, 50 U.C. Davis L. Rev. 1181 (2017); Evan C. Zoldan, Corpus Linguistics and the Dream of Objectivity, 50 Seton Hall L. Rev. 401 (2019). The Supreme Court has examined patterns of word use through newspaper databases,40×40. Muscarello v. United States, 524 U.S. 125, 129 (1998). and state supreme courts have searched corpora including the Corpus of Contemporary American English (COCA).41×41. See, e.g., People v. Harris, 885 N.W.2d 832, 839 (Mich. 2016). The growing use of dictionaries and legal corpus linguistics is likely to continue.42×42. Just last year, corpus linguistic analysis appeared twice in opinions from Judge Thapar of the Sixth Circuit. Judge Thapar issued a concurring opinion that relied on his corpus linguistics analysis, see Wilson v. Safelite Grp., Inc., 930 F.3d 429, 444 (6th Cir. 2019) (Thapar, J., concurring in part and concurring in the judgment), and a recent order requested that parties provide supplemental briefing that includes an explanation of how the corpus of Founding-era American English bears on the questions presented, see Wright v. Spaulding, 939 F.3d 695, 700 n.1 (6th Cir. 2019).
The Supreme Court’s 2019 Term included a number of blockbuster interpretation cases, including whether the Eighth and Fourteenth Amendments permit states to abolish the insanity defense, Kahler v. Kansas, 140 S. Ct. 1021 (2020), and whether Title VII’s prohibition of discrimination “because of . . . sex,” 42 U.S.C. § 2000e-2(a)(1), prohibits discrimination against employees for being lesbian, gay, or transgender, Bostock v. Clayton County, 140 S. Ct. 1371 (2020). Corpus linguistics scholars filed a brief in the latter case. See Brief for Corpus-Linguistics Scholars Professors Brian Slocum, Stefan Th. Gries, and Lawrence Solan as Amici Curiae in Support of Employees, Bostock v. Georgia, 139 S. Ct. 1599 (2019) (No. 17-1618).
Yet, despite the enthusiasm surrounding dictionaries and legal corpus linguistics, there is surprisingly little work assessing what these tools actually do in legal interpretation. Although the use of dictionaries and legal corpus linguistics seems to grow more sophisticated,43×43. See, e.g., Lee & Mouritsen, supra note 14, at 830–36; Lawrence M. Solan & Tammy Gales, Corpus Linguistics as a Tool in Legal Interpretation, 2017 BYU L. Rev. 1311, 1337–41. their reliability has never been rigorously assessed.44×44. By comparison, other approaches to judicial interpretation have been addressed by empirical studies. See generally Abbe R. Gluck & Lisa Schultz Bressman, Statutory Interpretation from the Inside — An Empirical Study of Congressional Drafting, Delegation, and the Canons: Part I, 65 Stan. L. Rev. 901 (2013) [hereinafter Gluck & Bressman, Part I]; Lisa Schultz Bressman & Abbe R. Gluck, Statutory Interpretation from the Inside — An Empirical Study of Congressional Drafting, Delegation, and the Canons: Part II, 66 Stan. L. Rev. 725 (2014). There are two published experimental surveys about originalism. See Donald L. Drakeman, What’s the Point of Originalism?, 37 Harv. J.L. & Pub. Pol’y 1123 (2014); Jamal Greene, Nathaniel Persily & Stephen Ansolabehere, Profiling Originalism, 111 Colum. L. Rev. 356 (2011). These are fascinating studies, but neither tests the reliability of originalist/textualist methodology. Instead, they focus on questions such as why people are originalists. See, e.g., Greene et al., supra, at 359. There are important critiques of these methods from external theoretical perspectives,45×45. See generally Ethan J. Herenstein, The Faulty Frequency Hypothesis: Difficulties in Operationalizing Ordinary Meaning Through Corpus Linguistics, 70 Stan. L. Rev. Online 112 (2017); Carissa Byrne Hessick, Corpus Linguistics and the Criminal Law, 2017 BYU L. Rev. 1503; Victoria Nourse, Picking and Choosing Text: Lessons for Statutory Interpretation from the Philosophy of Language, 69 Fla. L. Rev. 1409 (2017); Brian G. Slocum, Ordinary Meaning and Empiricism, 40 Statute L. Rev. 13 (2019). but we might also take an internal perspective, considering whether these methods succeed on their own terms. Theories relying on these tools typically assume that dictionaries and legal corpus linguistics reliably reflect ordinary meaning,46×46. For a demonstration of the use of corpus linguistics, see Lee & Mouritsen, supra note 14, at 836–45. However, that demonstration is not necessarily a demonstration of a reliable method of corpus linguistics. what “the ordinary user of the English language might understand,”47×47. Kavanaugh, supra note 25, at 2150 n.158. but the question remains: Is this assumption true?
This Article develops a novel method to test dictionaries and legal corpus linguistics. This method provides evidence about the reliability of these tools and, by extension, of the theories that rely on them. Insofar as a legal interpretive theory relies upon dictionary definitions or patterns of word usage, the study here also provides evidence about the success of such legal theories. Part I outlines the background to these debates and legal theories in which ordinary meaning analysis is significant. Part II surveys the tools that provide interpretive evidence for those theories, including the consultation of dictionary definitions and patterns of linguistic usage across corpora.
Parts III and IV consider the accuracy and reliability of these putative measures of ordinary meaning. One reason that these tools have not yet been tested is that such a test may seem impossible, particularly when we are considering original meaning in some historical time period. How can we evaluate the accuracy of an eighteenth-century dictionary or even a corpus linguistics search from the 1980s without facts about the way in which the contested term was actually understood at the time (that is, without the best data about its original ordinary meaning)? However, what we can do is evaluate whether modern uses of contemporary dictionaries and legal corpus linguistics reflect terms’ modern meanings. If a tool (for example, dictionary use) performs poorly in modern interpretation, so long as there are no historically distinguishing factors (for example, reasons that use of an eighteenth-century dictionary would be more reliable in historical interpretation than use of a modern dictionary is in modern interpretation), this fact gives us some evidence that the method is also unreliable in estimating original meaning.
To answer this central question — do dictionaries and legal corpus linguistics reliably reflect ordinary meaning? — the Article presents a robust experimental investigation of dictionaries and legal corpus linguistics. Experimental studies of ordinary people, law students, and United States judges all indicate that the way people understand ordinary terms and phrases (for example, “vehicle” or “carrying a firearm”) varies systematically from what a dictionary definition or relevant legal corpus linguistics’ usage data would indicate about the meaning.
After presenting and analyzing a range of experiments, the Article identifies ten noteworthy results and elaborates on their implications. Judges, law students, and ordinary people were strikingly similar in their ordinary conceptual judgments, use of dictionaries, and use of legal corpus linguistics. But within each group there existed considerable disagreement. For example, subjects within each group were divided on whether a canoe is a vehicle.
Legal corpus linguistics users tended to identify narrow, prototypical examples (for example, a car is a vehicle) rather than nonprototypical examples (for example, an airplane is a vehicle), while dictionary users tended in the opposite direction, supporting more extensive judgments (for example, a pair of roller skates is a vehicle). As such, the verdicts of legal corpus linguistics and dictionary use often diverged from each other.
Moreover, both legal corpus linguistics and dictionary use diverged from ordinary meaning — at a conservative divergence rate between 20–35%. That is, across all the levels of expertise — ordinary people, law students, and judges — the data suggest that relying on just a dictionary definition or selection of legal corpus linguistics data would lead users to the wrong judgment about “ordinary meaning” fairly often, once in every three to five cases. And that conservative rate may not tell the whole story. The experiments included a number of relatively easy categorizations, such as whether a car is a vehicle. Insofar as real legal decisions often concern comparatively harder categorizations, such as whether an airplane is a vehicle, it is also instructive to consider the maximum error rate: For example, what percentage of judges using dictionary or legal corpus linguistics methods evaluated the hardest interpretive question incorrectly? Across all levels of expertise, relying on a dictionary definition or corpus linguistics data led 80–100% of users to the incorrect verdict.
For theories committed to the notion of a single “ordinary meaning” that determines legal outcomes across a range of cases and contexts, the results suggest that dictionaries and legal corpus linguistics — two central tools of discovering ordinary meaning — are unreliable in interpretation. This shifts the argumentative burden to those who rely on these tools to provide both a principled defense of their use and an empirical demonstration of how error can be avoided. Without such an account and demonstration, it is hard to take seriously claims that these tools should be used by judges to determine “ordinary meaning” or that these tools deliver objective, restrained, or nondiscretionary interpretive outcomes.
Moreover, the results help identify common fallacies in the use of dictionaries and legal corpus linguistics. For instance, consider what this Article calls the “Nonappearance Fallacy” — namely, the (false) claim that absence of a usage from a large corpus indicates that the usage is not part of the ordinary meaning. One might be surprised to learn that in some modern corpora, there are no airplanes referred to as “vehicles.” Although it is tempting to conclude from this that airplanes are not included in the ordinary meaning of “vehicles,” the experiments show this to be too quick: the majority of ordinary people, law students, and judges evaluated airplanes to be vehicles. The broader insight underlying these critiques is that ordinary meaning diverges from ordinary use. Although courts and commentators sometimes conflate these concepts, there is a crucial distinction between ordinary meaning and what is typically spoken and recorded (for example, in a corpus).
For theories uncommitted to, or even skeptical of, the notion of a single “ordinary meaning,” the results illuminate two different criteria that are often relevant in assessments of the meaning of legal texts: a more extensive criterion and a more narrow, prototypical criterion. For example, an extensive criterion indicates that airplanes, canoes, and even drones are vehicles, while a more prototypical criterion indicates that these entities are not vehicles (only cars and similar entities are vehicles). The findings show that dictionaries or legal corpus linguistics sometimes help clarify one of these criteria — often dictionaries point to the extensive criterion and legal corpus linguistics to the prototypical one — but a hard legal-philosophical question remains: Which of these should serve as a criterion in legal interpretation? Good reasons underlie both criteria, especially the fact that ordinary people may understand the relevant concepts with both. As such, the results suggest that dictionary definitions, legal corpus linguistics, or even other more scientific measures of meaning may not be equipped in principle to deliver simple and unequivocal answers to inquiries about the ordinary meaning of law. Instead, in hard cases, legal interpreters will likely have to look beyond the dictionary and corpus — to the legal text’s context, history, and purpose.
* Assistant Professor of Law, Georgetown University Law Center; Research Affiliate, ETH Zurich Center for Law & Economics. Many thanks to the editors of the Harvard Law Review; to audiences at the Association for American Law Schools, Experimental Methods in Legal Scholarship, American Law and Economics Association, and Law & Corpus Linguistics at BYU conferences; and for thoughtful comments from Ellen Aprill, Elliott Ash, Stefan Bechtold, Clark Cunningham, Jesse Egbert, Richard Elliott, William Eskridge, Ed Finnegan, Tammy Gales, Stephen Galoob, Tobias Gesche, Abbe Gluck, Neal Goldfarb, Stefan Gries, Ivar Hannikainen, David Hoffman, Daniel Keller, Josh Kleinfeld, Markus Kneer, Josh Knobe, Issa Kohler-Hausmann, Anita Krishnakumar, Thomas Lee, Tom Lyon, Jamie Macleod, Stefan Magen, Daniel Markovits, John Mikhail, Stephen Mouritsen, Victoria Nourse, Karolina Prochownik, Kevin Reuter, Martin Schonger, Scott Shapiro, Dan Simon, Brian Slocum, Larry Solan, Roseanna Sommers, Alexander Stremitzer, Mila Versteeg, Pascale Willemsen, Gideon Yaffe, and Evan Zoldan.