Equal protection doctrine is bound up in conceptions of intent. Unintended harms from well-meaning persons are, quite simply, nonactionable.1 This basic predicate has erected a high bar for plaintiffs advancing equal protection challenges in the criminal justice system. Notably, race-based challenges on equal protection grounds, which subject the state to strict scrutiny review, are nonetheless stymied by the complexities of establishing discriminatory purpose. What’s more, the inability of courts and scholars to coalesce on a specific notion of the term has resulted in gravely inconsistent applications of the doctrine.2
States’ increasing use of algorithmic systems raises longstanding concerns regarding prediction in our criminal justice system — how to categorize the dangerousness of an individual, the extent to which past behavior is relevant to future conduct, and the effects of racial disparities.3 Integration of algorithmic systems has been defined by ambivalence: while they are touted for their removal of human discretion,4 they also readily promote and amplify inequalities — for example, through their consideration of protected characteristics and their interaction with existing systems tainted by legacies of inequality.5 Furthermore, algorithms, especially those incorporating artificial intelligence (AI), may operate in ways that are opaque, unpredictable, or not well understood. As commentators have demonstrated, intent-based notions inherent in equal protection jurisprudence are ill-suited to artificial intelligence.6
In response, scholars have presented a host of proposals to address the use of race by algorithmic systems, including colorblindness, affirmative action, outright prohibition, and the extension of effects-based statutory regimes.7 Many of these suggestions entail a sharp departure from the current discriminatory purpose requirement in favor of an effects-based framework attuned to the need to provide redress for unjust treatment.8 A doctrinal shift of this nature would be normatively beneficial, but is unlikely in the near future given the composition of the Supreme Court.9
This Note argues that transitioning to a test rooted primarily in evaluation of effect for equal protection challenges to the use of algorithmic risk assessment systems (RASs) would not present as significant a shift as described. Courts already rely extensively on effects when determining whether a discriminatory purpose exists in jury selection and voting cases.10 This Note proposes that the Supreme Court extend its current jurisprudence in these contexts to the use of algorithmic RASs in sentencing. Part I describes algorithmic RASs and how current intent-based notions are ill-suited to them. Part II illustrates how equal protection doctrine may incorporate primary evaluation of effects for algorithms. Part III demonstrates how an effects-plus framework may resolve equal protection challenges to algorithmic RASs. A brief conclusion follows.
I. Wrestling a Square Peg into a Round Doctrinal Hole
A. Increasingly Artificial Governance
Algorithmic systems are already a consequential tool of the administrative, judicial, and carceral state. Their most controversial application is arguably in the criminal justice system, in which courts have upheld their use in risk assessment for policing, bail, charging, probation, sentencing, and parole determinations.11 Throughout the country, RASs weigh factors correlated with risk to inform decisions relating to diverse categories of crimes.12 For example, pretrial risk assessment relies on actuarial data to determine a defendant’s risk of failing to appear in court and committing new criminal activity before trial.13 Post-adjudication risk assessment evaluates factors related to recidivism to inform the imposition of sentences, supervision, and treatment.14
Despite their increasing use and significance, algorithmic systems are not widely understood. Algorithmic RASs commonly used in sentencing are simple from a technological standpoint;15 most resemble automated checklists in that they estimate risk by assigning weight to a limited number of risk factors.16 Nevertheless, the operation of today’s RASs is practically inscrutable to defendants given access hurdles: such systems are often developed by private-sector companies that operate with limited oversight and raise the shield of trade secret protection.17 For example, the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS), an RAS in use in many jurisdictions, has a proprietary algorithm.18
Algorithms vary in their complexity and transparency: while structured algorithms may make carefully constrained decisions based on pre-programmed rules in a transparent manner,19 machine-learning programs — which fall under the umbrella of artificial intelligence — learn from past data and experience, potentially through opaque processes with few or no rules constraining their ability to predict.20 Some portend the wide use of technologically sophisticated AI processes,21 which may consider myriad variables and evaluate not only risk but also associated social cost.22 As a current example, although it primarily operates through standard regression models, COMPAS incorporates machine learning.23
The task of RASs, whether AI-enabled or not, in some ways is nothing different from what judges do every day.24 However, their perceived scientific legitimacy may entrench their use, despite uncertainties and contrary evidence regarding their precision and prediction gains.25 Moreover, algorithmic systems may be slower to change than human decisionmakers.26 This resulting inertia is especially problematic when data the systems work from are distorted.
All of this is not to overlook the benefits of algorithmic systems in terms of consistency and uniformity.27 Algorithmic RASs claim to be objective in reducing the opacity and subjectivity inherent in human decisionmaking, especially when issues of unconscious human bias are taken into account.28 In part for these reasons, scholars note that subjective risk assessment, as performed by judges making decisions based on their past experience and judgment, has the potential to exacerbate inequalities associated with prediction.29 A simple algorithm may at least explain itself through statistical measures of the correlations or weights associated with variables under consideration.30
B. The Standard Approach
Algorithmic systems do not easily fit the contours of modern equal protection doctrine, which provides two relevant paths for a plaintiff to establish a prima facie equal protection claim. First, in Washington v. Davis,31 the Supreme Court established that constitutional equal protection challenges to government action yielding a disproportionate racial impact must first encounter the filter of discriminatory purpose.32 A claim may pass upon only circumstantial evidence of effect plus intent to discriminate; the burden then shifts to the state to provide a nondiscriminatory justification.33 Since Davis, the Court has rejected the incorporation of a pure effects test. In Personnel Administrator of Massachusetts v. Feeney,34 it required a plaintiff to provide evidence of subjective intent to harm and rejected evidence that an action was taken “merely ‘in spite of . . . ’ its adverse effects upon an identifiable group.”35 Second, the Feeney Court laid out an alternate path that does not mandate a showing of discriminatory purpose: actions that are expressly conditioned on a racial classification or an “obvious pretext,” “regardless of purported motivation,” are “presumptively invalid,” as they “in themselves supply a reason to infer antipathy.”36 The state may rebut with a sufficient nondiscriminatory rationale, although in practice, this presumption is hard to combat.37
The discriminatory purpose requirement has erected a practically insurmountable burden of persuasion for plaintiffs and produced a considerable chilling effect on equal protection claims.38 It has accordingly engendered a host of criticisms: opponents have condemned the ease of hiding discriminatory motives and the myriad incentives to do so.39 Scholars have emphasized that the element of intent does not align with modern conceptions of unconscious bias and racism.40 They have also questioned the relevance of motive where disparate harm has been established.41 Such concerns justify a shift to an impact standard.
C. The Doctrinal Challenge
Beyond the justifications for an impact standard presented in the prior section, the argument for abandoning evaluation of intent is pronounced in the case of algorithmic systems for several reasons.
1. The Lack of Intent. — The focus of the discriminatory purpose requirement on intent is inapposite to algorithmic systems. Algorithms do not possess intent.42 Disparate impact rather stems from inevitable data biases or the intentional choices of their creators or the judges applying them.43 However, one may not discern relevant intent by evaluating the motives of either human creator or judge because neither is sufficiently responsible for the decisions of algorithmic RASs. Relevant decisionmaking for equal protection challenges to RASs includes the features selected and factors weighed by an algorithm, which are outside the control of a human judge. The intent of human creators manifests in decisions regarding the selection of training data, the definition of the optimization problem, and, in some cases, the application of features in the optimization problem.44 However, with complex AI systems, designers may not be responsible for the exact ways in which their tools operate or capable of explaining a system’s choices.45
2. The Obscuring Effect of Proxies. — Determining whether a factor is “an obvious pretext” for a protected characteristic is a hard problem. Algorithmic systems largely avoid the sort of overt consideration of protected characteristics that would render them presumptively invalid. RASs do not explicitly incorporate race as a factor.46 They instead base predictions on proxies: facially neutral factors — for example, diagnoses, marital status, neighborhood features, childhood experiences, and family criminal background47 — that tend to be strongly correlated with race.48 Discriminatory purpose doctrine’s probe for explicit classifications thus inadvertently insulates consideration of protected characteristics, the constitutionality of which is a matter of debate.49
3. The Countervailing Goal of Accuracy. — In a generalized way, AI systems aim to maximize accuracy — a model whose predictions do not reflect reality is worthless.50 However, the promotion of accuracy as the be-all and end-all is likely to inflict harm on protected classes: statistically “fair” associations driving prediction may reflect racially distorted rates of inputs among protected groups.51 The primary challenge is that the most significant variables from the standpoint of predictive accuracy tend to be correlated with protected attributes such as race due to unequally applied criminal justice practices — consider arrest rates or detention practices skewed against African Americans.52 Moreover, fairness metrics abound, leading to contrary evaluations of the impact of a specific algorithm.53 For example, in 2016, ProPublica, an investigative news organization, reported that COMPAS produced racially disparate results.54 Such results could be blamed on the tool’s reliance on variables highly correlated with race, including criminal history and neighborhood crime rates.55 But other scholars show that the choice of fairness metric impacts evaluations of COMPAS’s results — under some metrics, there aren’t clear racial imbalances.56 All of this amounts to considerable ambiguity with respect to whether an individual is being discriminated against through algorithms.
4. Court Application. — The failure of the discriminatory purpose requirement is demonstrated by State v. Loomis,57 which centered on a due process challenge to a trial court’s use of COMPAS.58 In upholding the use of COMPAS,59 the Wisconsin Supreme Court underwent a similar process in trying to discern the intent of the system as would be required under current interpretations of the discriminatory purpose requirement. For the court, the task amounted to an evaluation of whether the system was utilizing a protected classification and how that classification factored into a human judge’s analysis.60 The court rejected the defendant’s challenge to COMPAS’s consideration of gender in sentencing, finding the goal of “promot[ing] accuracy” sufficed as a neutral explanation.61 Moreover, the court held there was insufficient evidence that the sentencing court applying COMPAS had considered gender or rendered a decision solely on the basis of group characteristics.62 The court thus found the retention of a human decisionmaker alleviated concerns about the lack of an individualized decision.63 The Wisconsin Supreme Court’s analysis did not take into account the nuances of the workings of algorithms described above. As discussed in the next Part, its approach is neither mandated under equal protection analysis nor normatively beneficial for the criminal justice system.
II. Emphasizing Effect in the Evaluation of Algorithms
State and federal courts alike have varied in their application of the discriminatory purpose requirement to various categories of cases.64 Notably, jury selection and political apportionment present two limited contexts in which the Supreme Court has deviated from the general requirement of intent and left room for greater weighing of effects.65 This Part argues that courts should extend the approach to RASs given the similar interests underlying these cases.
A. Fit with Established Exceptions
An effects-plus framework is already embedded into the fabric of the discriminatory purpose requirement. As explained above, Davis and its progeny suggest that something more than sole evaluation of the effects of a facially neutral law is necessary to equal discriminatory purpose. In Davis, in the context of police officer exams, this something was intent. But current doctrine does not rigidly frame intent as the only element capable of fulfilling the requirement: Davis explicitly referenced the contexts of jury selection and voting, where the Court has accepted evidence of disparate impact without inquiry into motive when coupled with an additional element.66 Of course, intent is the default second element and, as Davis and its progeny indicate, a departure must be justified by something other than the difficulty of bringing an equal protection claim. Professor Daniel Ortiz suggests that this justification stems from the fundamental rights at stake in these domains.67 This Part argues that the interests and challenges implicated by algorithmic RASs justify a different second element.
1. Jury Selection. — The concern of keeping racial bias out of jury selection has led the Court to minimize the requisite showing for a prima facie antidiscrimination claim. This is borne out through challenges to jury selection approaches and the prosecution’s use of peremptory strikes. First, evidence of disparate impact in jury selection approaches has permitted an inference of discriminatory purpose based on a totality of the circumstances and without evaluation of intent.68 Davis acknowledged that “systematic exclusion of eligible jurymen of the proscribed race” may prove discriminatory purpose but noted that this fact “does not in itself make out an invidious discrimination forbidden by the [Equal Protection] Clause.”69 A second element is needed. For example, in finding unconstitutional the use of the “key-man” system of selecting juries in Castaneda v. Partida,70 the Court quoted Davis to explain that individuals could make out a prima facie claim with evidence that a racial group has been underrepresented in the jury selection process plus evidence that the process is susceptible to abuse.71 Once a plaintiff has made out a prima facie case of discrimination, the burden shifts to the government to demonstrate a permissible, race-neutral justification beyond a lack of a discriminatory motive.72
Second, under the Batson v. Kentucky73 framework, the Court has struck down states’ use of peremptory challenges based on little more than a showing of disparate impact.74 In Batson, the Court held that it was unconstitutional to exercise peremptory challenges to remove prospective jurors solely on the basis of race and that a defendant could establish purposeful discrimination “solely on the facts concerning . . . selection in his case.”75 It affirmed Davis’s notion that the “invidious quality” of government action claimed to be discriminatory “must ultimately be traced to a racially discriminatory purpose,”76 but established a three-step burden-shifting framework that heavily weighed effects. First, a defendant must make a prima facie showing that relevant circumstances raise an inference that a peremptory challenge was exercised on the basis of race.77 This requirement may be satisfied by demonstrating disproportionate impact.78 Second, upon that showing, the burden shifts to the state to offer a race-neutral explanation beyond denial of a discriminatory motive.79 Third, the court determines whether the defendant has shown “purposeful discrimination.”80 In its most recent case addressing Batson challenges, the Court reaffirmed its holding that clear statistical evidence of egregious disparate effect, coupled with little more, stands in for the intent requirement.81
Algorithmic RASs similarly justify departure from the element of intent. Both applications are paramount in the trial right — the fact-finding of juries complements the work of a neutral judge in determining a sentence. Yet, with both juries and algorithmic systems, concerns of bias, both conscious and unconscious, can infringe upon the individual liberty interest in a fair trial. Both represent contexts that may pose challenges in the discernment of pretext — our criminal justice system has retained peremptory strikes to protect government discretion, but it is often difficult to prove that the proffered reasons are inaccurate, implausible, or false.82 These concerns, which justify a standard that is not solely based on notions of intent for jury selection, also play out in the case of algorithmic systems. The difficulty of proving the intent of prosecutors ratchets up to impossibility in the case of AI.
Moreover, the potential use of proxies in both jury selection and RASs suggests that assessment of impact is similarly vital in both contexts. Given prohibitions against the explicit consideration of race in jury selection, decisionmakers may rely on strongly correlated attributes to arrive at the same outcomes.83 This is all the more likely given the lack of meaningful constraints to limit the discretion of decisionmakers in their selection of jurors.84 Similar concerns are present in the context of algorithmic RASs: with respect to sentencing, decisionmakers are already afforded wide latitude in determining factors to consider.85 This discretion is amplified by the lack of meaningful oversight of RAS creators and the lack of rules governing some prediction processes.86 The discretion of algorithmic designers is even more troubling than that of judges: whereas judges are subject to minimal measures of accountability,87 the criminal justice system has not imposed a similar level of accountability on private companies designing RASs.88 Thus, the interests of justice justify the omission of any requirement to prove discriminatory intent.
In the context of both jury selection and algorithmic RASs, the scope of the parties impacted extends beyond the persons whose rights are immediately at stake. Discriminatory jury selection impacts not only a defendant’s equal protection rights but also those of the excluded juror.89 Professor Brooks Holland notes that courts may show “greater ambivalence” when only a “guilty” defendant’s rights are at issue — even though an equal protection claim does “not necessarily bear on the defendant’s factual guilt or innocence” — than when the rights of “innocent” victims such as jurors are implicated.90 However, as Holland notes, the harm of racial bias in jury selection reaches beyond excluded jurors to “society” and the “rule of law.”91 In the same way, racial bias in algorithmic RASs has great potential — in conjunction with uncertain predictions of what a defendant may or may not do in the future — to further weaken confidence in sentencing.92
As demonstrated by the ease with which the government has been able to meet its burden of showing a race-neutral reason in jury selection cases,93 allowing greater incorporation of effect may not in practice diminish the government’s ability to evade searching review of equal protection challenges. However, reliance on impact is still beneficial because it allows plaintiffs to reach Batson’s step two — the state’s burden. This allocation of the burden of persuasion more appropriately accounts for informational disparities and the greater role of the state in authorizing the use of algorithms. Individuals who have identified a disparate impact are less likely to be able to present concrete evidence regarding algorithmic operation than the state actors responsible for the use of such algorithmic systems. Under this framework, the onus will more appropriately fall on the state, which will be forced to reckon with the propriety of its prediction problem and the application to a defen-dant. Even if such a task does not necessarily force the state to cease use of a system creating a disparate impact, it will incentivize the state to understand the workings of its systems, avoid overly complicated approaches, and opt for explainable, transparent operations.
2. Political Apportionment. — In the context of vote dilution, the Court has accepted evidence of disparate impact as evincing discriminatory purpose when coupled with evidence of past discrimination, even when there is no clearly identifiable decisionmaker involved.94 In rejecting a racial vote dilution challenge to at-large municipal elections in City of Mobile v. Bolden,95 a plurality of the Justices invoked Feeney to emphasize that an Equal Protection Clause violation requires a showing of “purposeful” discrimination that cannot be met with evidence of disparate impact alone.96 However, the Court swiftly changed course, as Congress amended the Voting Rights Act at least in part in response to criticism of Bolden.97 Two years later, in a challenge to another at-large voting system that allegedly discriminated against Black residents in Rogers v. Lodge,98 the Court reaffirmed intent as a primary focus.99 However, it made clear that equal protection doctrine analyzes how the electoral system functions. The Court found that the intent element was satisfied by “an aggregate of . . . factors”100 that more closely evidenced impact, including the fact that no Black people had ever been elected and the history of past discrimination against Blacks in the political process.101 As the dissents noted, the Court did not identify the relevant decisionmakers or their intent102 — a result seemingly inconsistent with Davis. The Court’s opinion in fact quoted Davis in asserting that “an invidious discriminatory purpose may often be inferred from the totality of the relevant facts, including the fact, if it is true, that the law bears more heavily on one race than another.”103
The Court has also been willing to infer a legislature’s overreliance on a scrutinized classification, like race, based on resulting demographic impact or bizarre district shape. Professor Heather Gerken notes that the Supreme Court has referred to these cases as involving “facially neutral classifications” and yet declined to require proof of intent as necessitated by Davis and its progeny.104 For example, in Shaw v. Reno,105 the Court held in favor of White voters in their challenge to majority-minority districts without requiring proof of discriminatory purpose; the Court instead inferred impermissible reliance on race from the bizarre shape of the challenged district.106 Similarly, while more modern redistricting cases have followed the rationale of Davis in requiring a showing that race was the predominant factor,107 courts have embraced plaintiffs’ use of statistical evidence of impact where states have used district-drawing software.108 For example, in Bush v. Vera,109 the Court struck down Texas’s redistricting scheme, which was the product of the use of sophisticated software that drew lines in ways that suggested race was a proxy for political affiliation.110
The use of algorithmic RASs resembles the voting context in the challenge of discerning improper motives. Implicit incorporation of race is expected in both: section 5 of the Voting Rights Act requires consideration of race, albeit in a limited manner;111 similarly, algorithmic RASs implicitly incorporate factors highly correlated with race. In evaluating whether consideration of a factor by an algorithmic RAS is improper, courts face a doctrinal challenge similar to discerning the collective intent of a legislature. Both contexts lack one relevant human decisionmaker for evaluation of intent. Both also feature severe informational disparities between individuals challenging government action and the state itself — the proprietary nature of many algorithms creates a similar roadblock as do political factors insulating motives for the shaping of districts. The subtlety of these relationships counsels against entangling courts in the determination of the significance of a specific variable.
The difficulties of discerning intent justify evaluation of results such as the shape of a district or the ensuant marginalization of a racial group in sentencing. The Court’s willingness to primarily consider effects should thus extend to critical consideration of RASs, which are replete with facially neutral practices that may work to deprive defendants of a fundamental liberty interest. As redistricting cases illustrate, the potential to impact a range of decisions, such as the need to correct representation flowing from unconstitutionally drawn district lines, is not fatal: an injunction against a redistricting scheme may carry at least as much impact as one against use of an RAS.112
B. Distinguishing Algorithms from Traditional Sentencing Challenges
Standing most directly in the way of plaintiffs seeking to make out a prima facie claim for racial discrimination in sentencing is the precedent of McCleskey v. Kemp.113 There, the Court dismissed a defendant’s challenge to Georgia’s death penalty statute on the basis of a comprehensive statistical study conducted by Professors David C. Baldus, Charles Pulaski, and George Woodworth (the Baldus study),114 which showed that the imposition of capital punishment was strongly correlated with the race of a defendant and the race of a victim.115 In so doing, the Court rejected the capacity of statistics to provide circumstantial evidence of a discriminatory individual sentencing decision.116 Plaintiffs have accordingly struggled to overcome McCleskey’s bar in all but the starkest cases of discrimination.117
McCleskey was wrongly decided and should be overruled.118 Commentators have opined that the Court’s reasoning “misconstrued . . . the effectiveness of statistical analyses.”119 McCleskey stands out in that it involved a significant individual liberty interest — life itself, in the context of capital punishment, in which the Court has applied greater procedural protections and heightened liability relative to other sentencing cases120 — and yet saddled the individual with the high burden of showing actual motivation.121 This section demonstrates that the McCleskey decision does not necessarily bar development of a framework emphasizing impact for algorithmic RASs.122 Although McCleskey addresses a plaintiff’s burden in the sentencing context, the use of RASs is sufficiently distinct to justify a different approach.
The McCleskey Court undoubtedly was reluctant to accept disparate impact as the sole basis of an equal protection challenge. First, while the Court accepted the validity of the Baldus study,123 it applied Feeney’s construction of discriminatory purpose in requiring a showing of intent by the legislature to enact or maintain the death penalty statute specifically “because of an anticipated racially discriminatory effect.”124 Statistical evidence yielding a generalized inference of class-based harm would not suffice.125 The Court distinguished capital sentencing decisions from Title VII and jury selection in that, in the latter contexts, “the statistics relate to fewer entities, and fewer variables are relevant to the challenged decisions.”126 It emphasized that the discretion of prosecutors, juries, judges, and others involved in crafting a sentence is “essential” so as to demand “exceptionally clear proof” of abuse before escalation to strict scrutiny.127 Second, the opinion stressed the need to provide the state an effective chance to rebut an inference of discriminatory intent: in jury selection cases, the decisionmaker may explain the disparity, whereas the state lacks the same opportunity to defend the prosecutor’s and jury’s decisions to seek and impose the death penalty, since their decisions may have been made years prior and the jury cannot generally be called to testify.128 Third, the Court credited the presence of the “legitimate and unchallenged explanation” that Georgia law permitted capital punishment.129
Claims based on algorithmic RASs differ from McCleskey’s claim in a few important ways. For example, McCleskey’s claim “thr[ew] into serious question” the legitimacy of a broad range of sentences and convictions — “the principles that underlie our entire criminal justice system.”130 While a claim based solely on the Baldus study would impugn a host of cases made by different decisionmakers — “every actor in the Georgia capital sentencing process”131 — challenges to RASs implicate one uniform decisionmaker and thus involve only cases in which a particular algorithm was used.132 RASs also offer the opportunity for explanations of factors. Though the proprietary nature of some RASs undermines the depth of their explanations, such opportunity is at least as present as in jury selection cases.133 In sum, while McCleskey jeopardizes the fate of equal protection challenges in the sentencing context generally, the application of algorithmic RASs differs in important ways that justify departures from McCleskey’s approach.
III. Application of an Effects-Plus Framework
It remains to be seen what an effects-plus framework may look like in a challenge to the use of algorithmic RASs. This Part contends, in line with other scholarly proposals,134 that an effects-plus framework resolves the tension inherent in equal protection doctrine regarding algorithms. The framework would collapse current doctrine’s two paths for a plaintiff to establish a prima facie claim. Under the first, a plaintiff may show a system’s explicit classification based on race, which is presumptively invalid.135 However, given the reliance of AI on proxies, such a showing should not be conclusive of invalidity, as it would dismiss or obscure the impact of inevitable proxies that feature in algorithmic prediction.
Thus, regardless of the presence of an explicit racial classification, a plaintiff must first demonstrate disparate impact, as evidenced through statistical evidence of racial disparity.136 As the necessary second element, the plaintiff must demonstrate that the outcome predicted by an RAS is susceptible to racially biased analysis. This second element aligns with showing susceptibility to abuse in the jury selection cases and a history of discrimination in the voting cases because it addresses the likelihood of bias or manipulation in an action implicating fundamental liberty interests. This element acknowledges that predictions of an outcome that happens more readily among a certain class of people, based on past data that may reflect racial inequities, will increase racial disparities.137 A plaintiff could challenge specific factors used in the RAS or critique the manner of use by a sentencing judge.
That brings us to the state’s burden to provide a neutral (non-discriminatory) justification.138 This burden exists in both the Davis framework and cases arising in the jury selection and voting contexts. Relating to peremptory strikes, the Court has acknowledged that allowing any relevant reason to constitute a permissible, race-neutral justification would counteract the goal of Batson.139 In practice, myriad reasons have been found to constitute non-pretextual rationales for dismissing a juror.140 The promotion of accuracy, which drives statistical systems, ostensibly presents such a justification. Indeed, where the predicted outcome is the actual outcome of interest, the promotion of accuracy may suffice as a reasonable justification. However, where there is a mismatch, the burden of persuasion should require the state to provide a reason that isn’t solely the general promotion of accuracy; accuracy with respect to an imperfect proxy that may be racially biased is not a useful aim for the criminal justice system. Requiring the government to articulate a reason for discriminatory effect in these cases may incentivize state actors and algorithmic developers to ensure they can understand why an algorithm makes decisions ahead of time and limit the implementation of opaque sentencing algorithms where they cannot do so.
The approach can be illustrated by a race-based equal protection challenge to the system at issue in Loomis. The plaintiff would have to first show a disparate effect involving the use of an RAS — for example, that COMPAS yielded disparate racial impact in sentencing. The plaintiff would next have to show that the outcome predicted (future arrest) is susceptible to abuse through racial bias in the input data.141 Given the racial distortions in arrest data, the state would have to justify its inclusion of the factor with a reason that does not amount solely to a desire to make accurate predictions of arrest, such as the relevance of an individual’s age or maturity to their likelihood of recidivism or the value of particular neighborhood characteristics in demonstrating specific support systems. The mere retention of a human judge overseeing the process would not suffice to meet the state’s burden if the decisionmaker in any way consulted the algorithmic outputs.
Evidence of discriminatory motive can also factor into this framework — for example, where a plaintiff seeks to show that creators have designed a program so as to make discrimination based on protected attributes probable. This motive may be evidenced through the inclusion of facially neutral factors known to cause disparate racial impact. For example, Judge Calabresi has argued that the legislature’s enactment of a law while aware of the racial impact of a significant sentencing disparity would violate the Equal Protection Clause.142 In a similar vein, where neighborhood characteristics have been shown to cause a discriminatory impact, their inclusion — even when framed in facially neutral ways —may evince a harmful bias in the system and a corresponding intent by human designers to effect racial discrimination. But this showing, which is difficult due to the proprietary nature of RASs, is not required.
The approach outlined above seeks to permit the use of algorithmic RASs in criminal justice only where their benefits in terms of uniformity and traceability outweigh potential harms. As an illustration, the use of risk assessment as a diagnostic tool to evaluate the effectiveness of interventions that weigh on the ability of individuals to reoffend or attend future court events is appropriate and adequately safeguards individual autonomy.143 But, in sentencing, which is inherently centered on punishment for past behavior,144 prediction processes are based on uncontrollable factors and the costs of error are substantial. It is thus important to take into account the disconnect between outcomes of interest, which will often relate to the purposes of punishment, and the predicted proxies, which are often tainted by historic inequality. This approach is intended to work in tandem with proposed efforts to correct informational imbalances in sentencing, such as disclosure regimes that allow individuals to access the workings of programs utilized to make decisions.145
Conclusion
It is a truism widely acknowledged that the best predictor of future behavior is past behavior. However, the use of algorithmic RASs pushes us to weigh the costs of this principle in the face of biased predictions and illusory remedial schemes. It is increasingly challenging to raise an equal protection claim successfully given the discriminatory purpose requirement. Even in domains such as jury selection and voting, where an effects-plus framework does not require evidence of intent, litigants have faced an uphill battle as state actors are afforded deference in their explanations of facially neutral actions that produce discriminatory impact. A similar framework for algorithmic RASs would likely run into these hurdles.
Consequently, a doctrinal shift for the discriminatory purpose requirement — one that would incorporate a broader conception of intent or embrace primary evaluation of impact — is normatively beneficial.146 But doctrinal shifts more often than not occur in increments. This Note showcases an incremental step in the development of a more robust equal protection framework that better responds to the workings of technology. Alongside transparency and disclosure proposals, it also aims to provide plaintiffs a wider opportunity to assert challenges in the face of discriminatory practices in the criminal justice system and thus realign the doctrine with its goal to protect against the harms of discrimination, whether conscious or not.