Introduction
Ignorance isn’t bliss; a system rooted in faulty assumptions and unproven hunches risks terrible mistakes and cannot represent the best interests of justice. Professor Andrew Crespo’s Systemic Facts: Toward Institutional Awareness in Criminal Courts starts from this simple yet critically important insight to offer a compelling policy prescription for the United States’s failing criminal courts.1 If these courts are to rule fairly and preside effectively over the functions of the criminal justice system, Crespo argues, they need to operate against some baseline of accuracy. Their decisions must be rooted in facts — facts that have a foundation in lived experience and the community that the lawyers and judges of the court purport to represent. In Crespo’s account, courts routinely ignore these facts, although the facts are known and readily accessible.2 They are the facts that judges confront every day — the race of criminal defendants, the accuracy of police predictions, and the rates of crimes committed in a given neighborhood.3 Crespo argues that by internalizing, embracing, and incorporating these facts into their decisions and case management, criminal courts may provide a check on a range of procedural abuses. These “systemic facts” may assure a degree of accuracy in the criminal justice system and ensure that judges, rather than police or prosecutors, are able to contextualize each case.
This Response, which proceeds in two Parts, considers the promises and limitations of Crespo’s proposed approach to systemic facts. In Part I, I argue that Crespo presents a compelling case for the importance of systemic factfinding to the task of criminal court judges. If, as a range of scholars has argued, criminal courts are increasingly serving a quasi-administrative function, then shouldn’t they at least be administrating accurately? Systemic Facts provides a novel account of how — with comparatively little institutional reform — courts might begin to serve as more effective administrators. However, in Part II, I also argue that Crespo’s account largely takes for granted the unmitigated benefit of more data and more information. I argue that Crespo’s account understates the indeterminacy of systemic facts and the ways in which data rely on interpreters and interpretations. To the extent our criminal justice system already suffers from the assumptions and biases of judges and other official actors, granting those same actors the ability to interpret a wealth of data or “facts” need not dictate a move toward greater justice or greater accuracy. Rather, it might allow the same actors to reach the same decisions, supported by the same underlying assumptions, but bolstered by a powerful new dataset. Ultimately, systemic factfinding might highlight the problems with criminal courts, but it would not necessarily provide a vehicle for reform or redress.
I. Facts Superseding Assumptions
Systemic Facts is framed as an intervention into what Crespo dubs the “New Administrativist” literature.4 The New Administrative scholars have recognized that criminal courts operate on a transactional basis — processing large numbers of defendants, in a one-off fashion, almost always without trial.5 In such a context, these scholars have concluded that courts are ill-suited to the task at hand, a task that is essentially administrative in nature and that requires broader knowledge of and engagement with the criminal justice system.6 By embracing the insights and institutions of administrative law, these scholars have advocated for administrative solutions to systemic problems.7 Perhaps police or law enforcement agencies should self-regulate, or perhaps some other executive agency should be tasked with oversight. But the solutions that these scholars have proposed (and that Crespo responds to) presume that courts are not the ideal regulatory body to prevent official misconduct.8
Crespo posits a similar world in which the transactional model hampers systemic engagement. But he concludes that reformers need not look beyond courts themselves to find the best actors to preside over the administration of criminal justice.9 Rather, Crespo contends that courts already have access to numerous facts — arrest locations and demographics, accuracy of police testimony, and prosecutors’ disclosure of Brady10 material — that should allow courts to perform the sort of systemic management that the New Administrativists seek.11 Additionally, Crespo contends that court-centric solutions actually may be more politically feasible than a delegation of authority to some other regulatory agency.12
This intervention is noteworthy on its own terms, but Systemic Facts should also be seen as a contribution to a different scholarly tradition: the call for empiricism (or perhaps empirical accuracy) in judicial decisionmaking. At its core, the article is a part of a larger realist project that seeks to challenge judicial assumptions by describing both the factual circumstances and the effects of judicial decisionmaking.13 Courts reach decisions based on assumptions about facts on the ground and about how their rulings affect criminal defendants, police, and prosecutors. But many of these assumptions have no empirical or authoritative support. A judge may conclude that a stop was justified because it occurred in a “high crime area,” but the law does not require — and judges rarely provide — any evidence of a neighborhood’s crime rates.14 So we are left with an opinion and a legal rule that purport to rely on empirical evidence, but we have no such evidence against which to test the court’s conclusion.
In Crespo’s view, decisions unsupported by systemic facts have two potential flaws: (1) their reasoning is not transparent, and (2) the decisions themselves might not be based on accurate information. As to the first objection, Crespo is undoubtedly correct, and his argument for courts’ use of systemic facts appears responsive. Writing a decade and a half ago, Professors Tracey Meares and Bernard Harcourt “call[ed] for a mode of judicial decision-making and academic debate that treats social scientific and empirical assessment as a crucial element in constitutional decision-making, thereby making criminal procedure decisions more transparent.”15 As Meares and Harcourt put it, “[j]udicial decisions that address the relevant social science and empirical data are more transparent in that they expressly articulate the grounds for factual assertions and, as a result, more clearly reflect the interpretive choices involved in criminal procedure decision-making.”16 How can an appellate court make its decision if the record below has factual gaps? And how can policymakers and activists determine if rules (or judges)17 need changing if they cannot assess the basis for a ruling? Opinions and decisions supported by evidence would allow observers (litigants, politicians, and activists) to appreciate the judge’s normative views. That is, when a judge “shows her work” or provides the underlying facts or data on which she relies, it is easier to determine whether she reached the “right” result.
As to Crespo’s second objection, certainly accuracy is an issue. Recent scholarly critiques of criminal procedure jurisprudence make a strong case that the assumptions driving (or at least justifying) many decisions are not supported by facts on the ground.18 For example, Professor Seth Stoughton has shown that the Court frequently relies on erroneous assumptions about the nature of policing.19 Similarly, Professor Shima Baradaran Baughman has provided empirical evidence to undermine the nexus between drugs and violence that the Court has used to justify expansive police powers in drug cases.20 Encouraging courts to collect and apply systemic facts might help judges reach more accurate results by forcing them to test their assumptions against facts on the ground. Or, at the very least, the tension between those facts and a judge’s assumptions might be clearer to the parties or to reviewing courts.21
Crespo’s insights are valuable, and Systemic Facts makes an important contribution to the growing literature on systemic reform of criminal justice policy. By emphasizing courts’ access to a wealth of information about policing and prosecution, he has provided a key insight into the literature on judicial empiricism in constitutional criminal law. But, as I discuss in the next Part, I think he is too optimistic about the effectiveness of more facts when it comes to changing outcomes and is insufficiently focused on the ideological and structural barriers that might not make courts the best actors to interpret systemic facts.
II. Assumptions Superseding Facts
Legal and factual analysis cannot be divorced from a judge’s priors.22 In noting the limitations of systemic facts, Crespo acknowledges that “systemic factfinding cannot replace the exercise of careful judgment when it comes to the core normative questions at the heart of the criminal justice system, nor will it necessarily force judges to abandon any preexisting views they might hold.”23 This observation does not necessarily undermine the importance of systemic facts. But it does raise the question of why courts are the best place for applying (rather than collecting) systemic facts. That is, Crespo’s analysis seems to turn on a belief that it is factual ignorance, rather than normative preferences or priors, that is to blame for courts’ shortcomings. I’m not sure that he is right. And, at the very least, a decision to grant courts broader administrative authority should require a closer inspection of the forces and preferences that shape criminal courts.24
What are “criminal courts”? And how can we trust them to know, digest, and apply facts? As a project, Systemic Facts shows a desire to take institutions seriously.25 Crespo reminds us that too much scholarship focuses on appellate courts, ignoring the realities of the trial courts that shape the criminal justice system.26 This critique strikes me as accurate, but if this institutional turn is important, then we need a thicker descriptive account of the institutions themselves. That is, we need to know the identities of the individual decisionmakers, as well as their incentives. In order to decide that courts are better situated than other actors (namely police or prosecutors) to weigh interests of “liberty” and “security,”27 we need to understand courts’ relationships to police, prosecutors, and other criminal justice actors.
Such an understanding would require that we excavate the inner workings of criminal courts. Crespo explains that his “references to a ‘criminal court’ are to the court as an institution, and thus generally encompass both the judges who adjudicate cases and the administrative personnel who assist the court in carrying out its judicial function.”28 As lawyers, we generally use “courts” and “judges” interchangeably, but doing so undersells the human components of the judicial system. Therefore, Crespo’s observation is an important one and a potential curative for this common elision. But his definition operates as a footnoted aside rather than a broader lens through which we might view his analysis and proposal.
If we are to trust “courts” with the impressive array of tasks that Crespo would assign them, then I think we need to know more about courts. Or, in other words, we need to recognize that courts, as institutions, are made up of different actors with different interests, incentives, and priors. Data do not speak for themselves, so we would need to understand who would be interpreting the raw data in Crespo’s proposed world of systemic factfinding.29
From Professor Malcolm Feeley’s The Process Is the Punishment30 to Professor Issa Kohler-Hausmann’s Managerial Justice,31 scholars of law and society have taken on the task of exposing the hidden players and dynamics in criminal courts. While I am not suggesting that Systemic Facts should have been an ethnographic work, I want to emphasize (as Crespo acknowledges) that “courts” are not monolithic. There are many actors and decisionmakers whom we might identify as part of the “court” and who might be implicated in the identification and interpretation of systemic facts. Members of the probation department, law clerks, staff attorneys, and docket clerks might have very different backgrounds, training, and incentives from the judges they assist.32 In a court system that adopted Crespo’s proposal, these actors might well be important components of the judicial apparatus charged with amassing and digesting systemic facts.
Further, what of the judges themselves? If “courts” are primarily “judges,” then Crespo’s proposal is primarily an embrace of an expansive vision of the judicial role. In Crespo’s proposal, judges would adopt a quasi-inquisitorial model as keepers and interpreters of systemic facts. While Crespo is careful to rebut concerns about a dearth of adversarialism, he has much less to say about the politics and incentives of the individual judges.33 Judicial elections, he notes, might make us worry about the ability of courts to serve as impersonal and neutral fiduciaries of fact.34 Yet, outside of some discussion of judges’ technical capacity to interpret data, Crespo scarcely mentions judges’ personal characteristics. Are they former prosecutors or defense attorneys?35 Is it common for former civil litigators to preside over criminal courts? Do judges tend to come from the same communities as the defendants appearing before them?
These questions fall outside of the scope of Systemic Facts, but in order to assess the merits of Crespo’s proposal, I think we need to answer them. Assuming that a judge’s experiences or priors shape the way that she interprets a given case, her interpretation (and perhaps collection) of systemic facts would also rest on these assumptions and biases.36 That is, systemic facts about criminal adjudication might be contingent on a range of facts about criminal courts themselves. Information on the composition of the federal bench is readily accessible,37 but biographical and demographic information about state trial court judges should also be a part of the discussion.38 If one of the primary purposes of Systemic Facts is to argue that courts themselves should find, interpret, and apply facts, then judges’ priors, incentives, assumptions, and training require greater attention.39 Indeed, Crespo, like the New Administrativists and most scholars of criminal law, takes as a starting premise that the criminal justice system has “gone ‘seriously awry’” — that courts are failing to reign in abuses and misconduct and that the rules in place are not resulting in a fair administration of justice.40 If that is the case, then courts have already proven inadequate when faced with the task of reigning in a system run amok. Crespo’s answer seems to be that if we give courts more information they will see the error of their ways. Maybe. And, maybe, as Crespo acknowledges, such an outcome is more likely when a judge is presented with a particularly glaring set of facts.41
I agree with Crespo that more facts might help judges realize their implicit biases, but the beauty of Crespo’s policy proposal is also a major issue for this line of argument: the reason that courts (rather than agencies) are well suited to apply systemic facts is that courts already know those facts. That is, on a daily basis, courts see who is arrested, hear officers testify, and preside over voir dire conducted by the same prosecutors. While the transactional nature of criminal adjudication might obscure the cumulative impact of individual decisions, it is important to recognize that criminal courts have had the opportunity to learn from these facts. And it is not clear that they have taken advantage of this opportunity. Put simply, my concern with Systemic Facts is that I think it is too optimistic in its assumption that the collection and explicit reliance on facts will be effective in significantly reducing the problems posed by incentives of political economy, by ideological priors, and by assumptions about how the world works.
Certainly, one response to these critiques would be to throw up our hands and reject empiricism and the prospect of using systemic facts to achieve systemic reform. But that would be wrong.42 These facts (and empirical evidence generally) can play an important role in improving the transparency and function of the criminal justice system. In this respect, I agree with Crespo. The criminal justice system has dramatic and tragic real-world effects. I agree that it would be a normative good if the rules and decisions that undergird the system had some support in the facts on the ground. To the extent that they do not, I agree that it is important that we know, so that we can work to determine the best solutions. But more facts do not necessarily mean more justice or fairer outcomes.
Indeed, the excellent defenses of judicial empiricism articulated by Harcourt, Meares, and (at points in Systemic Facts) Crespo himself focus on empiricism’s promises and limits in improving criminal adjudication. Harcourt has argued that social scientists should “orient the empirical project specifically on the assumptions embedded in law and policy making. Rather than use the research to draw law and policy inferences, use the research to expose the assumptions about human behavior that . . . underlie the law and policy proposals.”43 And, Crespo suggests that systemic facts can “excavate, illuminate, and crystallize those normative questions, allowing courts to engage them more directly and with greater nuance and precision.”44
To me, this is the power of empiricism, and it is one of the reasons that I applaud Systemic Facts. Crespo has gone to great lengths to acknowledge what systemic facts cannot do.45 And that is critically important. But if what systemic facts can do is highlight normative preferences and demonstrate judicial assumptions, then how would they allow courts to do all the tasks that the New Administrativists have identified?46 And how would they necessarily make substantive outcomes better, rather than merely more transparent? If we want to right the ship of U.S. criminal justice, we need a systemic reevaluation and a means of checking and reforming actors systemically. Certainly, understanding what is wrong and increasing transparency so that more people can appreciate what is wrong are important steps. But without challenging the values, assumptions, and “pathological politics”47 that shape criminal courts, it is not clear how more facts — systemic, or otherwise — will allow courts to police themselves and serve as the organs of criminal justice reform.
* Climenko Fellow and Lecturer on Law, Harvard Law School. Thanks to Andrew Crespo for generously welcoming these critiques and to the Harvard Law Review for inviting my Response and for editorial guidance. Thanks also to Susannah Barton Tobin for helpful comments and suggestions.