Harvard Law Review Harvard Law Review Harvard Law Review

Harvard Law Review Forum

Values and Assumptions in Criminal Adjudication

Response to Andrew Manuel Crespo, Systemic Facts: Toward Institutional Awareness in Criminal Courts


Ignorance isn’t bliss; a system rooted in faulty assumptions and unproven hunches risks terrible mistakes and cannot represent the best interests of justice. Professor Andrew Crespo’s Systemic Facts: Toward Institutional Awareness in Criminal Courts starts from this simple yet critically important insight to offer a compelling policy prescription for the United States’s failing criminal courts.1×1. Andrew Manuel Crespo, Systemic Facts: Toward Institutional Awareness in Criminal Courts, 129 Harv. L. Rev. 2049 (2016). If these courts are to rule fairly and preside effectively over the functions of the criminal justice system, Crespo argues, they need to operate against some baseline of accuracy. Their decisions must be rooted in facts — facts that have a foundation in lived experience and the community that the lawyers and judges of the court purport to represent. In Crespo’s account, courts routinely ignore these facts, although the facts are known and readily accessible.2×2. See generally id. pts. II–III, 2065–101. They are the facts that judges confront every day — the race of criminal defendants, the accuracy of police predictions, and the rates of crimes committed in a given neighborhood.3×3. See generally id. Crespo argues that by internalizing, embracing, and incorporating these facts into their decisions and case management, criminal courts may provide a check on a range of procedural abuses. These “systemic facts” may assure a degree of accuracy in the criminal justice system and ensure that judges, rather than police or prosecutors, are able to contextualize each case.

This Response, which proceeds in two Parts, considers the promises and limitations of Crespo’s proposed approach to systemic facts. In Part I, I argue that Crespo presents a compelling case for the importance of systemic factfinding to the task of criminal court judges. If, as a range of scholars has argued, criminal courts are increasingly serving a quasi-administrative function, then shouldn’t they at least be administrating accurately? Systemic Facts provides a novel account of how — with comparatively little institutional reform — courts might begin to serve as more effective administrators. However, in Part II, I also argue that Crespo’s account largely takes for granted the unmitigated benefit of more data and more information. I argue that Crespo’s account understates the indeterminacy of systemic facts and the ways in which data rely on interpreters and interpretations. To the extent our criminal justice system already suffers from the assumptions and biases of judges and other official actors, granting those same actors the ability to interpret a wealth of data or “facts” need not dictate a move toward greater justice or greater accuracy. Rather, it might allow the same actors to reach the same decisions, supported by the same underlying assumptions, but bolstered by a powerful new dataset. Ultimately, systemic factfinding might highlight the problems with criminal courts, but it would not necessarily provide a vehicle for reform or redress.

I. Facts Superseding Assumptions

Systemic Facts is framed as an intervention into what Crespo dubs the “New Administrativist” literature.4×4. Id. at 2057–59. The New Administrative scholars have recognized that criminal courts operate on a transactional basis — processing large numbers of defendants, in a one-off fashion, almost always without trial.5×5. See id. at 2054–57. In such a context, these scholars have concluded that courts are ill-suited to the task at hand, a task that is essentially administrative in nature and that requires broader knowledge of and engagement with the criminal justice system.6×6. See id. By embracing the insights and institutions of administrative law, these scholars have advocated for administrative solutions to systemic problems.7×7. See id. at 2057–58. Perhaps police or law enforcement agencies should self-regulate, or perhaps some other executive agency should be tasked with oversight. But the solutions that these scholars have proposed (and that Crespo responds to) presume that courts are not the ideal regulatory body to prevent official misconduct.8×8. See id. at 2057.

Crespo posits a similar world in which the transactional model hampers systemic engagement. But he concludes that reformers need not look beyond courts themselves to find the best actors to preside over the administration of criminal justice.9×9. See id. at 2059–65. Rather, Crespo contends that courts already have access to numerous facts — arrest locations and demographics, accuracy of police testimony, and prosecutors’ disclosure of Brady10×10. Brady v. Maryland, 373 U.S. 83 (1963). material — that should allow courts to perform the sort of systemic management that the New Administrativists seek.11×11. See generally Crespo, supra note 1, at pts. II–III, 2065–101. Additionally, Crespo contends that court-centric solutions actually may be more politically feasible than a delegation of authority to some other regulatory agency.12×12. See id. at 2063–64.

This intervention is noteworthy on its own terms, but Systemic Facts should also be seen as a contribution to a different scholarly tradition: the call for empiricism (or perhaps empirical accuracy) in judicial decisionmaking. At its core, the article is a part of a larger realist project that seeks to challenge judicial assumptions by describing both the factual circumstances and the effects of judicial decisionmaking.13×13. See, e.g., Shima Baradaran, Drugs and Violence, 88 S. Cal. L. Rev. 227 (2015); Tracey L. Meares, Three Objections to the Use of Empiricism in Criminal Law and Procedure — And Three Answers, 2002 U. Ill. L. Rev. 851; Tracey L. Meares & Bernard E. Harcourt, Foreword: Transparent Adjudication and Social Science Research in Constitutional Criminal Procedure, 90 J. Crim. L. & Criminology 733 (2000); Sonja B. Starr, Evidence-Based Sentencing and the Scientific Rationalization of Discrimination, 66 Stan. L. Rev. 803 (2014); Seth W. Stoughton, Policing Facts, 88 Tul. L. Rev. 847 (2014). Courts reach decisions based on assumptions about facts on the ground and about how their rulings affect criminal defendants, police, and prosecutors. But many of these assumptions have no empirical or authoritative support. A judge may conclude that a stop was justified because it occurred in a “high crime area,” but the law does not require — and judges rarely provide — any evidence of a neighborhood’s crime rates.14×14. See Crespo, supra note 1, at 2079–85. So we are left with an opinion and a legal rule that purport to rely on empirical evidence, but we have no such evidence against which to test the court’s conclusion.

In Crespo’s view, decisions unsupported by systemic facts have two potential flaws: (1) their reasoning is not transparent, and (2) the decisions themselves might not be based on accurate information. As to the first objection, Crespo is undoubtedly correct, and his argument for courts’ use of systemic facts appears responsive. Writing a decade and a half ago, Professors Tracey Meares and Bernard Harcourt “call[ed] for a mode of judicial decision-making and academic debate that treats social scientific and empirical assessment as a crucial element in constitutional decision-making, thereby making criminal procedure decisions more transparent.”15×15. Meares & Harcourt, supra note 13, at 735. As Meares and Harcourt put it, “[j]udicial decisions that address the relevant social science and empirical data are more transparent in that they expressly articulate the grounds for factual assertions and, as a result, more clearly reflect the interpretive choices involved in criminal procedure decision-making.”16×16. Id. How can an appellate court make its decision if the record below has factual gaps? And how can policymakers and activists determine if rules (or judges)17×17. More on this point in Part II. need changing if they cannot assess the basis for a ruling? Opinions and decisions supported by evidence would allow observers (litigants, politicians, and activists) to appreciate the judge’s normative views. That is, when a judge “shows her work” or provides the underlying facts or data on which she relies, it is easier to determine whether she reached the “right” result.

As to Crespo’s second objection, certainly accuracy is an issue. Recent scholarly critiques of criminal procedure jurisprudence make a strong case that the assumptions driving (or at least justifying) many decisions are not supported by facts on the ground.18×18. See, e.g., Lee Epstein et al., Foreword: Testing the Constitution, 90 N.Y.U. L. Rev. 1001, 1024 (2015); Max Minzner, Putting Probability Back into Probable Cause, 87 Tex. L. Rev. 913, 915 (2009). For example, Professor Seth Stoughton has shown that the Court frequently relies on erroneous assumptions about the nature of policing.19×19. See generally Stoughton, supra note 13. Similarly, Professor Shima Baradaran Baughman has provided empirical evidence to undermine the nexus between drugs and violence that the Court has used to justify expansive police powers in drug cases.20×20. See generally Baradaran, supra note 13. Encouraging courts to collect and apply systemic facts might help judges reach more accurate results by forcing them to test their assumptions against facts on the ground. Or, at the very least, the tension between those facts and a judge’s assumptions might be clearer to the parties or to reviewing courts.21×21. See Crespo, supra note 1, at 2099–101.

Crespo’s insights are valuable, and Systemic Facts makes an important contribution to the growing literature on systemic reform of criminal justice policy. By emphasizing courts’ access to a wealth of information about policing and prosecution, he has provided a key insight into the literature on judicial empiricism in constitutional criminal law. But, as I discuss in the next Part, I think he is too optimistic about the effectiveness of more facts when it comes to changing outcomes and is insufficiently focused on the ideological and structural barriers that might not make courts the best actors to interpret systemic facts.

II. Assumptions Superseding Facts

Legal and factual analysis cannot be divorced from a judge’s priors.22×22. See, e.g., James B. Atleson, Values and Assumptions in American Labor Law 1–4 (1983); Gary Minda, Boycott in America: How Imagination and Ideology Shape the Legal Mind 8–13 (1999); Richard D. Parker, “Here, The People Rule”: A Constitutional Populist Manifesto 53–115 (1994); Kathryn Abrams, Some Realism About Electoralism: Rethinking Judicial Campaign Finance, 72 S. Cal. L. Rev. 505, 509 (1999); Nancy S. Ehrenreich, Pluralist Myths and Powerless Men: The Ideology of Reasonableness in Sexual Harassment Law, 99 Yale L.J. 1177, 1218 (1990); Dan M. Kahan, The Secret Ambition of Deterrence, 113 Harv. L. Rev. 413, 428 (1999); Richard L. Revesz, Environmental Regulation, Ideology, and the D.C. Circuit, 83 Va. L. Rev. 1717, 1766–67 (1997). In noting the limitations of systemic facts, Crespo acknowledges that “systemic factfinding cannot replace the exercise of careful judgment when it comes to the core normative questions at the heart of the criminal justice system, nor will it necessarily force judges to abandon any preexisting views they might hold.”23×23. Crespo, supra note 1, at 2114; see also Meares & Harcourt, supra note 13, at 793–94, 797. This observation does not necessarily undermine the importance of systemic facts. But it does raise the question of why courts are the best place for applying (rather than collecting) systemic facts. That is, Crespo’s analysis seems to turn on a belief that it is factual ignorance, rather than normative preferences or priors, that is to blame for courts’ shortcomings. I’m not sure that he is right. And, at the very least, a decision to grant courts broader administrative authority should require a closer inspection of the forces and preferences that shape criminal courts.24×24. In raising these questions, I don’t mean to suggest that some other actors or institutions necessarily would be better at this task than courts. That is, priors or normative preferences may shape the ways that judges interpret or collect data, but they would also shape the ways in which administrative or executive actors would interpret and collect such data.

What are “criminal courts”? And how can we trust them to know, digest, and apply facts? As a project, Systemic Facts shows a desire to take institutions seriously.25×25. See Crespo, supra note 1, at 2056 (“[T]he discussion that follows is concerned . . . [with] criminal courts’ institutional structure, design, and capacity.”). Crespo reminds us that too much scholarship focuses on appellate courts, ignoring the realities of the trial courts that shape the criminal justice system.26×26. See id. at 2052–53. This critique strikes me as accurate, but if this institutional turn is important, then we need a thicker descriptive account of the institutions themselves. That is, we need to know the identities of the individual decisionmakers, as well as their incentives. In order to decide that courts are better situated than other actors (namely police or prosecutors) to weigh interests of “liberty” and “security,”27×27. See id. at 2061. we need to understand courts’ relationships to police, prosecutors, and other criminal justice actors.

Such an understanding would require that we excavate the inner workings of criminal courts. Crespo explains that his “references to a ‘criminal court’ are to the court as an institution, and thus generally encompass both the judges who adjudicate cases and the administrative personnel who assist the court in carrying out its judicial function.”28×28. Id. at 2054 n.8. As lawyers, we generally use “courts” and “judges” interchangeably, but doing so undersells the human components of the judicial system. Therefore, Crespo’s observation is an important one and a potential curative for this common elision. But his definition operates as a footnoted aside rather than a broader lens through which we might view his analysis and proposal.

If we are to trust “courts” with the impressive array of tasks that Crespo would assign them, then I think we need to know more about courts. Or, in other words, we need to recognize that courts, as institutions, are made up of different actors with different interests, incentives, and priors. Data do not speak for themselves, so we would need to understand who would be interpreting the raw data in Crespo’s proposed world of systemic factfinding.29×29. Crespo suggests that courts might require “expert assistance” to process and interpret data, see id. at 2053, 2082, which would introduce yet another set of actors (that is, the “experts”) whose interpretation would in turn depend on their own priors, assumptions, and views of the criminal justice system.

From Professor Malcolm Feeley’s The Process Is the Punishment30×30. Malcolm M. Feeley, The Process Is the Punishment: Handling Cases in a Lower Criminal Court (1979). to Professor Issa Kohler-Hausmann’s Managerial Justice,31×31. Issa Kohler-Hausmann, Managerial Justice and Mass Misdemeanors, 66 Stan. L. Rev. 611 (2014). scholars of law and society have taken on the task of exposing the hidden players and dynamics in criminal courts. While I am not suggesting that Systemic Facts should have been an ethnographic work, I want to emphasize (as Crespo acknowledges) that “courts” are not monolithic. There are many actors and decisionmakers whom we might identify as part of the “court” and who might be implicated in the identification and interpretation of systemic facts. Members of the probation department, law clerks, staff attorneys, and docket clerks might have very different backgrounds, training, and incentives from the judges they assist.32×32. While it focuses on the federal system, recent research by political scientists has examined the political ideology of law clerks and what it does (and doesn’t) reflect about judicial politics. See generally Adam Bonica et al., The Political Ideologies of Law Clerks and their Judges (Univ. of Chi. Coase-Sandor Inst. for Law & Econ., Research Paper No. 756, 2016), http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2739478 [http://perma.cc/MLU8-48NS]. In a court system that adopted Crespo’s proposal, these actors might well be important components of the judicial apparatus charged with amassing and digesting systemic facts.

Further, what of the judges themselves? If “courts” are primarily “judges,” then Crespo’s proposal is primarily an embrace of an expansive vision of the judicial role. In Crespo’s proposal, judges would adopt a quasi-inquisitorial model as keepers and interpreters of systemic facts. While Crespo is careful to rebut concerns about a dearth of adversarialism, he has much less to say about the politics and incentives of the individual judges.33×33. While such a discussion is absent from Systemic Facts, other work by Crespo emphasizes judges’ priors and experiences, noting their importance to judicial decisionmaking. See Andrew Manuel Crespo, Regaining Perspective: Constitutional Criminal Adjudication in the U.S. Supreme Court, 100 Minn. L. Rev. (forthcoming 2016); see also Crespo, supra note 1, at 2113 n.283. Judicial elections, he notes, might make us worry about the ability of courts to serve as impersonal and neutral fiduciaries of fact.34×34. Crespo, supra note 1, at 2112–13. Yet, outside of some discussion of judges’ technical capacity to interpret data, Crespo scarcely mentions judges’ personal characteristics. Are they former prosecutors or defense attorneys?35×35. It is worth noting that studies suggest that judges with prosecutorial experience tend to outnumber those with criminal defense experience (particularly those with experience as public defenders). See, e.g., Gregory L. Acquaviva & John D. Castiglione, Judicial Diversity on State Supreme Courts, 39 Seton Hall L. Rev. 1203, 1235 tbl.10 (2009). Given that Crespo expresses skepticism about the ability of law enforcement to self-regulate, perhaps we should ask whether this skepticism about prosecutorial approaches to the criminal justice system evaporates when a prosecutor is nominated or elected to the bench. Is it common for former civil litigators to preside over criminal courts? Do judges tend to come from the same communities as the defendants appearing before them?

These questions fall outside of the scope of Systemic Facts, but in order to assess the merits of Crespo’s proposal, I think we need to answer them. Assuming that a judge’s experiences or priors shape the way that she interprets a given case, her interpretation (and perhaps collection) of systemic facts would also rest on these assumptions and biases.36×36. Cf. Linda Hamilton Krieger & Susan T. Fiske, Behavioral Realism in Employment Discrimination Law: Implicit Bias and Disparate Treatment, 94 Calif. L. Rev. 997, 1036–37 (2006) (describing studies showing that decisionmakers “spontaneously search for independent decision criteria consistent with their preference, and use those criteria to justify their choices to themselves and others” (citing Michael I. Norton et al., Casuistry and Social Category Bias, 87 J. Personality & Soc. Psychol. 817, 829 (2004))). That is, systemic facts about criminal adjudication might be contingent on a range of facts about criminal courts themselves. Information on the composition of the federal bench is readily accessible,37×37. See The Judicial Research Initiative at the Univ. of S.C., Attributes of Federal Judges, U. S.C., http://artsandsciences.sc.edu/poli/juri/attributes.htm [http://perma.cc/25KB-ATB5]. but biographical and demographic information about state trial court judges should also be a part of the discussion.38×38. For more on the background of attorneys and other members of the judiciary, see generally Adam Bonica et al., The Political Ideologies of American Lawyers (HKS Working Paper No. 15-049, 2015), http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2652706 [http://perma.cc/7GVC-PN8T]. If one of the primary purposes of Systemic Facts is to argue that courts themselves should find, interpret, and apply facts, then judges’ priors, incentives, assumptions, and training require greater attention.39×39. In addressing the question of courts’ and judges’ competency to interpret data, Crespo notes the rise of “empirical legal studies as a robust academic discipline.” See Crespo, supra note 1, at 2104. While he is certainly right that elite law schools have focused more research and resources on empirical methods and on scholars in these fields, it is less clear how much these developments would affect the criminal court judges that Crespo describes. Determining that influence would require us to know more about judges’ ages, education, and ties to legal academia. Indeed, Crespo, like the New Administrativists and most scholars of criminal law, takes as a starting premise that the criminal justice system has “gone ‘seriously awry’” — that courts are failing to reign in abuses and misconduct and that the rules in place are not resulting in a fair administration of justice.40×40. See id. at 2059 (quoting Barry Friedman & Maria Ponomarenko, Democratic Policing, 90 N.Y.U. L. Rev. 1827, 1829 (2015)). If that is the case, then courts have already proven inadequate when faced with the task of reigning in a system run amok. Crespo’s answer seems to be that if we give courts more information they will see the error of their ways. Maybe. And, maybe, as Crespo acknowledges, such an outcome is more likely when a judge is presented with a particularly glaring set of facts.41×41. See id. at 2113.

I agree with Crespo that more facts might help judges realize their implicit biases, but the beauty of Crespo’s policy proposal is also a major issue for this line of argument: the reason that courts (rather than agencies) are well suited to apply systemic facts is that courts already know those facts. That is, on a daily basis, courts see who is arrested, hear officers testify, and preside over voir dire conducted by the same prosecutors. While the transactional nature of criminal adjudication might obscure the cumulative impact of individual decisions, it is important to recognize that criminal courts have had the opportunity to learn from these facts. And it is not clear that they have taken advantage of this opportunity. Put simply, my concern with Systemic Facts is that I think it is too optimistic in its assumption that the collection and explicit reliance on facts will be effective in significantly reducing the problems posed by incentives of political economy, by ideological priors, and by assumptions about how the world works.

Certainly, one response to these critiques would be to throw up our hands and reject empiricism and the prospect of using systemic facts to achieve systemic reform. But that would be wrong.42×42. See Bernard E. Harcourt, Language of the Gun: Youth, Crime, and Public Policy, at xi–xii (2006). These facts (and empirical evidence generally) can play an important role in improving the transparency and function of the criminal justice system. In this respect, I agree with Crespo. The criminal justice system has dramatic and tragic real-world effects. I agree that it would be a normative good if the rules and decisions that undergird the system had some support in the facts on the ground. To the extent that they do not, I agree that it is important that we know, so that we can work to determine the best solutions. But more facts do not necessarily mean more justice or fairer outcomes.

Indeed, the excellent defenses of judicial empiricism articulated by Harcourt, Meares, and (at points in Systemic Facts) Crespo himself focus on empiricism’s promises and limits in improving criminal adjudication. Harcourt has argued that social scientists should “orient the empirical project specifically on the assumptions embedded in law and policy making. Rather than use the research to draw law and policy inferences, use the research to expose the assumptions about human behavior that . . . underlie the law and policy proposals.”43×43. Id. at xi; see also Meares & Harcourt, supra note 13, at 735 (“[U]se of empirical evidence will produce a clearer picture of the existing constitutional landscape and spotlight the normative judgments at the heart of criminal procedure cases.” (quoted in Crespo, supra note 1, at 2102 n.233)); Deborah Jones Merritt, Correspondence, Constitutional Fact and Theory: A Response to Chief Judge Posner, 97 Mich. L. Rev. 1287, 1287 (1999) (“[E]mpirical knowledge is most useful in unmasking the theoretical assumptions that undergird constitutional law . . . .”). And, Crespo suggests that systemic facts can “excavate, illuminate, and crystallize those normative questions, allowing courts to engage them more directly and with greater nuance and precision.”44×44. Crespo, supra note 1, at 2102.

To me, this is the power of empiricism, and it is one of the reasons that I applaud Systemic Facts. Crespo has gone to great lengths to acknowledge what systemic facts cannot do.45×45. See id. at 2101–02. And that is critically important. But if what systemic facts can do is highlight normative preferences and demonstrate judicial assumptions, then how would they allow courts to do all the tasks that the New Administrativists have identified?46×46. As noted previously, see supra note 24, these concerns and critiques apply equally to proposals to allow for police or prosecutorial self-regulation. And how would they necessarily make substantive outcomes better, rather than merely more transparent? If we want to right the ship of U.S. criminal justice, we need a systemic reevaluation and a means of checking and reforming actors systemically. Certainly, understanding what is wrong and increasing transparency so that more people can appreciate what is wrong are important steps. But without challenging the values, assumptions, and “pathological politics”47×47. See generally William J. Stuntz, The Pathological Politics of Criminal Law, 100 Mich. L. Rev. 505 (2001). that shape criminal courts, it is not clear how more facts — systemic, or otherwise — will allow courts to police themselves and serve as the organs of criminal justice reform.


* Climenko Fellow and Lecturer on Law, Harvard Law School.  Thanks to Andrew Crespo for generously welcoming these critiques and to the Harvard Law Review for inviting my Response and for editorial guidance.  Thanks also to Susannah Barton Tobin for helpful comments and suggestions.