Harvard Law Review Harvard Law Review Harvard Law Review

Technology

Economies of Surveillance

The full text of this Book Review may be found by clicking on the PDF link to the left.

[I]n or about December, 1910, human character changed.

— Virginia Woolf1×1. 1 Virginia Woolf, Mr. Bennett and Mrs. Brown, in Collected Essays 319, 320 (Leonard Woolf ed., 1966) (1950).

It was an age of innovation and a time of dispossession. By sweeping aside centuries-old patterns of economic and social organization, the Industrial Revolution grew productive capacities, eventually enabling higher living standards and greater human well-being.2×2. At the time, the “majority opinion of contemporary observers and students” saw no such gain. E.J. Hobsbawm, Laboring Men: Studies in the History of Labour 106 (2d ed. 1968). Historians have conducted a long, lively debate on the Industrial Revolution’s effect on living standards, but recent empirical work finds “almost a century of hard toil with little or no advance from a low base before [British workers and their families] really began to share in any of the benefits of the economic transformation.” Charles H. Feinstein, Pessimism Perpetuated: Real Wages and the Standard of Living in Britain During and After the Industrial Revolution, 58 J. Econ. Hist. 625, 652 (1998). But its birth pangs were experienced as unevenly as were its early rewards. Where its burdens fell hardest, their weight provoked “counter-moves”3×3. Karl Polanyi, The Great Transformation: The Political and Economic Origins of Our Time 79 (3d ed. 2001). challenging the new economic order. In rural Britain, for example, parliamentary acts of enclosure “drastically curtailed” common pasture and heath land used by the poor.4×4. Jane Humphries, Enclosures, Common Rights, and Women, 50 J. Econ. Hist. 17, 20 (1990). For many, the result was squalor and penury.5×5. Id. at 18–21; see also J.M. Neeson, The Opponents of Enclosure in Eighteenth-Century Northamptonshire, 105 Past & Present 114, 116–18 (1984) (explaining legislative dynamics of enclosure). Enclosure “destroyed the scratch-as-scratch-can subsistence economy of the poor.”6×6. E.P. Thompson, The Making of the English Working Class 217 (Vintage Books 1966) (1963). It “brok[e] the bond of mutual dependence between the master and his servant.”7×7. Id. at 224 (quoting Parliament, House of Commons, The Parish and the Union; or, the Poor and the Poor Laws Under the Old System and the New 90 (London, Charles Knight & Co. 1837)). Beginning in the Elham Valley of Kent in summer 1830, laborers led more than 1000 violent incidents, including the targeting of threshing machines and other agricultural machinery.8×8. Id.; see E.J. Hobsbawm & George Rudé, Captain Swing 195, 304–05 (1969). Despite not having halted the erosion of common property through enclosure, the “so-called ‘Swing Riots’”9×9. Carl J. Griffin, The Violent Captain Swing?, 209 Past & Present 149, 149 (2010). proved a “historical pivot” that politicized the rural population irreversibly.10×10. Id. at 150.

No less unsettled were the dark, satanic factories rising in cities and towns. The blunt clang of machinery became more familiar than the lark’s morning song. Vast industrial facilities proliferated, like the Soho Foundry and the metalworks of Smethwick.11×11. See Eric Roll, An Early Experiment in Industrial Organisation 2466 (1968). Overseers and owners thrust “the time-sheet, the time-keeper, the informers and the fines” upon men and women habituated to setting their own labor rhythms free from the clock’s confining chime.12×12. E.P. Thompson, Customs in Common 385 (1991). A newfangled “concept of industrial discipline” aimed to suborn recalcitrant human labor to the new economy’s inexorable rhythms.13×13. Sidney Pollard, Factory Discipline in the Industrial Revolution, 16 Econ. Hist. Rev. 254, 259 (1963). Instead, what ensued was “far-reaching conflict” over the terms of labor.14×14. Thompson, supra note 12, at 399. Violent resistance to new technologies, and to measures that reduced labor’s control over working conditions, followed.15×15. Immanuel Ness, Luddism and Machine Breaking, in The International Encyclopedia of Revolution and Protest 2133, 213334 (Immanuel Ness ed., 2009). Indeed, this continued to be the case well into the twentieth century. Labor unrest continued to be stoked by “tactics of scientific management (time study, task setting, efficiency payments, and so forth).”16×16. Mike Davis, The Stop Watch and the Wooden Shoe: Scientific Management and the Industrial Workers of the World, in Workers’ Struggles, Past and Present 83, 87 (James Green ed., 1983). Yet still the “prophets” of this new industrial capitalist age, men like Frederick Winslow Taylor and Frank and William Gilbreth, pressed relentlessly for “scientific management” — a project meant to drain industrial labor of its unpredictability,17×17. Bryan Palmer, Class, Conception and Conflict: The Thrust for Efficiency, Managerial Views of Labor and the Working Class Rebellion, 1903–22, 7 Rev. Radical Pol. Econ. 31, 34–35 (1975). but a project that also galvanized enduring conflicts over laborers’ felt autonomy and dignity.18×18. See id. at 41–44.

Two centuries later, amidst avulsions in everything from our planet’s climate to prevailing democratic structures, conflicts over dignity, autonomy, and predictability remain all too familiar, even if they occur in a distinct and newly dissonant register. In The Age of Surveillance Capitalism, Professor Shoshana Zuboff offers a far-reaching, and in some regards groundbreaking, account to explain why and to elucidate how these conflicts are evolving in the early twenty-first century. What people around the world are now enduring, she claims, is a convulsive rupture in economic and social life that is comparable in scale to the one precipitated by the first Industrial Revolution. Just as enclosure robbed the rural poor of a common heritage, today a novel economic form of “surveillance capitalism” works a “dispossession” (p. 100), a “domination” (p. 10), an “expropriation” (p. 128), and a “robbery” (p. 158) by usurping people’s control over the data associated with their lives.

The most salient difference Zuboff perceives between our experience and that of Cumbrian peasants and Lancaster weavers is the magnitude of our loss. Today, it is not mere surplus value, but “human nature” itself that is “scraped, torn, and taken for another century’s market project” (p. 94). Hence, akin to Virginia Woolf a century before her, Zuboff posits a radical discontinuity in “human character.”19×19. Woolf, supra note 1, at 320. But whereas Woolf explored the literary possibility of a more complex grasp of psychological interiority,20×20. See id. at 326–37. Zuboff channels the outrage of some more modern figures. We think her tone similar to that of the conservative commentator William F. Buckley or the Berkeley student activist Mario Savio.21×21. Roger Kimball, Introduction to Athwart History: Half a Century of Polemics, Animadversions, and Illuminations xxix (Linda Bridges & Roger Kimball eds., 2010); Mario Savio Dies at Age 53; Led ’60s Free-Speech Protests, Wash. Post, Nov. 8, 1996, at B6 (“There comes a time when the operation of the machine becomes so odious, makes you so sick at heart, that . . . [you must] put your bodies on the gears, and upon the wheels, upon the levers, upon all the apparatus. And you’ve got to make it stop.” (quoting Berkeley student activist Mario Savio)). Standing athwart the train of history as they did, Zuboff cries stop. She laments in unremitting terms the emergence of technologically mediated “one-way mirror[s]” that eviscerate the very possibility of interiority (p. 81), a development that “threatens to cost us our humanity” (p. 347).22×22. Emphasis has been omitted.

Zuboff’s dire diagnosis of what she terms “surveillance capitalism” culminates in a call to resistance. Hers is a rallying cry that echoes the resistance offered by laborers and artisans to enclosures of land and time two centuries ago. It is an effort to reimagine the anguished Marxian empathies of the early Industrial Revolution, but with crucial variations. Where once machine breaking and rick burning were the instruments of protest, today Zuboff is careful to avoid relying merely on civil society and the private resistance. It is not enough, on her view, to rely on “signal-blocking phone cases, false fingerprint prosthetics . . . [,] LED privacy visors . . . [, and] a ‘serendiptor app’ to disrupt any surveillance” (p. 489). Instead it is law — and more profoundly, the state — that stands in potent counterpoise to the threat of surveillance capitalism. In her reckoning, the state is a site of imperfect political action needful to vindicate human well-being and autonomy.

Such a mix of theory, exemplars, critique, and exhortation is no doubt heady stuff. It strikes a timely and relevant chord for lawyers and legal scholars. Ours is a historical moment at which worries about the technology economy, the health of civic life, and inequality haphazardly diffuse, a bit like Don DeLillo’s airborne toxic event, across domains of law ranging from antitrust to free speech to international security. Zuboff’s polemic crystallizes and offers vindication of that zeitgeist.

As we shall argue momentarily, certain elements of Zuboff’s account bristle with difficulties. But before broaching these concerns, we want to acknowledge her substantial contributions. To her large credit, Zuboff, an influential professor who has long taught at the Harvard Business School,23×23. On the influence of Zuboff’s 1988 book In the Age of the Smart Machine, see Andrew Burton-Jones, What Have We Learned from the Smart Machine?, 24 Info. & Org. 71 (2014). offers not just a new conceptual tool that encapsulates public anxiety about new technologies — “surveillance capitalism” — but also detailed and persuasive accounts of how specific privacy costs, fairness concerns, and psychological harms arise. Her book graphically stages the way in which new instruments for aggregating behavioral data and generating predictions reinforce the dominant position of a coterie of market participants capable of exploiting network effects. Her rich vein of examples and theory will no doubt stoke and sustain public anxiety for some time to come. And most relevant for our purposes, Zuboff also offers a valuable opportunity to think through important questions about the roles of law and the state caught in the ceaseless currents of technological innovation.24×24. Reviews of the book in nonacademic publications embody gratitude to Zuboff for crystallizing and confirming public terrors about technology. See Jennifer Szalai, O.K., Google: How Much Money Have I Made for You Today?, N.Y. Times (Jan. 16, 2019), https://nyti.ms/2RRhy4m [https://perma.cc/NX37-852F] (noting how the book speaks to the reviewer’s “reflexive discomfort”); see also James Bridle, The Age of Surveillance Capitalism by Shoshana Zuboff Review — We Are the Pawns, The Guardian (Feb. 2, 2019), https://www.theguardian.com/books/2019/feb/02/age-of-surveillance-capitalism-shoshana-zuboff-review [https://perma.cc/W4ML-6ULA].

For two reasons, we think that legal scholars in particular would do well to engage her theoretical positions and her examples. First, legal scholarship tends to be discrete in its focus and granular in its analysis when it comes to novel technological development. We myopically scrutinize a specific technology, such as social media platforms,25×25. E.g., Kate Klonick, The New Governors: The People, Rules, and Processes Governing Online Speech, 131 Harv. L. Rev. 1598 (2018). machine learning,26×26. E.g., Joshua A. Kroll et al., Accountable Algorithms, 165 U. Pa. L. Rev. 633 (2017). or the internet of things,27×27. E.g., Andrew Guthrie Ferguson, The Internet of Things and the Fourth Amendment of Effects, 104 Calif. L. Rev. 805 (2016). and try to understand how that phenomenon relates to existing legal templates. This work is valuable, even essential. But scholars and lawyers can miss the forest for the trees when they consider only parts rather than the integrated whole of the emerging data-driven economy. System-level effects, whether positive or negative, may be missed when discrete technologies or legal changes are analyzed in isolation. Gains or losses that spill over from one domain of human activity to another may be sliced out of the analytic frame. Without a clear sense of how discrete technologies are deployed, legal scholars are left with the feeling that they know something is happening, but they don’t know what it is.28×28. Cf. Bob Dylan, Ballad of a Thin Man, in Bob Dylan: The Lyrics 19612012, at 174, 174 (Simon & Schuster 2016) (2004) (“Because something is happening here/But you don’t know what it is.”). In contrast to such analytic pointillism, The Age of Surveillance Capitalism offers a coherent and synoptic account of how and to what effect new data-driven technologies are adopted. In the course of deploying and interrogating Zuboff’s perspective, legal scholars may gain leverage in considering broader-gauge questions about how technological change, business models, and social and cultural values interact with each other and the legal system.29×29. Cf. David Leonhardt, Book Review, The Forces that Are Killing the American Dream, N.Y. Times (Sept. 10, 2019), https://nyti.ms/32ApXLm [https://perma.cc/K8KG-6RWU] (“[In the late 1970s,] investors and their allies dismantled the corporate and government edifices that had done so much good — high wages, company research labs, rigorous regulation and redistributive taxation. Institutions were out. Transactions were in.”). This is so even if they disagree with many of Zuboff’s claims.

Second, when an idea chimes with the zeitgeist, it has the potential to powerfully shape public perceptions. This in turn can influence legal responses to emergent economic and technological phenomena. As one review of the book put it, Zuboff has captured the broadly held sense of something gone awry.30×30. See Bridle, supra note 24. She deftly dramatizes the public’s impression that the new data-driven economy is unheimlich: it is “so grotesque, so creepy, that it is almost impossible to see how anyone who really thinks about it lives with it.”31×31. Id. Harmony with present paranoia, of course, is no warrant of acuity. So when a narrative like Zuboff’s bids fair to colonize the political imagination and thereby shape the law’s long-term development, we have all the greater cause for close scrutiny.

A simple, modular logic in Zuboff’s argument facilitates close scrutiny. Her argument can be separated along three distinct disciplinary margins. First, she makes a historical claim about the novelty of surveillance capitalism. Second, she draws on a largely implicit, albeit hardly indefensible, skepticism of the connection between indicia such as revealed preference or observed economic growth on the one hand, and underlying social welfare, capaciously understood, on the other. With this skepticism in hand, she advances a normative claim about the surveillance economy’s harms. Third, she offers a prescriptive claim about the appropriate state response. If these elements of her argument are occluded at times, it is because Zuboff often drapes genuine insight in unwarranted hyperbole. Her tendencies in this regard lend the project a polemic quality. Zuboff’s argumentation is tainted by what Professor Richard Hofstadter once called the “paranoid style” of American politics, characterized by “heated exaggeration, suspiciousness, and conspiratorial fantasy.”32×32. Richard Hofstadter, The Paranoid Style in American Politics, Harper’s Mag., Nov. 1964, at 77, 77. Yet, as Hofstadter also observed: “[N]othing really prevents a sound program or demand from being advocated in the paranoid style.”33×33. Id. Removing the obscuring rhetorical veils not only helps expose what is genuinely new and relevant to legal scholars in The Age of Surveillance Capitalism, it also facilitates critique and reconstruction.

Zuboff keenly diagnoses flaws in capitalist logic and succeeds in cataloging surveillance capitalism’s manifold dangers. Yet serious problems arise along each of her three argumentative arcs. What Zuboff pervasively characterizes in Manichean terms34×34. See J. Eric Oliver & Thomas J. Wood, Conspiracy Theories and the Paranoid Style(s) of Mass Opinion, 58 Am. J. Pol. Sci. 952, 953 (2014) (finding that popular conspiracy theories tend to hold a “Manichean worldview” and depend on “unseen, intentional, and malevolent forces”). as the nefarious work of unseen manipulators turns out on closer scrutiny to be more complex and less amenable to facile legal intervention than she implies. Rather than embracing the zeitgeist’s “paranoid style” of thinking about the surveillance imaginary, we think that scholarship about our present technological and economic conjuncture should account for factors that get little or no airing in Zuboff’s text. We stress in particular the human benefits flowing from some of those new instruments;35×35. See infra Part III, pp. 1309–25. the permeable boundary between the state’s interests and corporate interests at the data-driven frontier; and the risks to individual liberty and democratic stability from expansive state control of surveillance capitalism.36×36. See infra Part IV, pp. 1326–35. By sorting through these difficulties, we hope in this Review to beat a path to a clearer and more plausible sense of law’s appropriate role — and the state’s place more generally — in this new era of data- and surveillance-based economies. Our analysis thus excavates flaws in Zuboff’s argument with the ambition of creating more persuasive and precisely targeted remedies to the real pathologies she flags.

Part I offers a summary of and commentary upon Zuboff’s main claims. We describe portions of Zuboff’s argument we find most compelling, identify unresolved questions, and also underscore continuities with deeper theoretical traditions. Part II interrogates the idea of a radical discontinuity between “surveillance capitalism” and its precursors. We explain here why we prefer the descriptive label “surveillance economies” instead of surveillance capitalism. Part III turns to the goods and harms that flow from new surveillance economies. Our accounting is hedged with more caveats than is Zuboff’s regarding the final tally of these goods and harms. Finally, Part IV theorizes afresh the role of the state in creating and checking those economies and their adverse spillovers. It offers three general principles to guide analysis and critique of that role.

 


*Justice, California Supreme Court; Herman Phleger Visiting Professor of Law and affiliated scholar, Freeman Spogli Institute for International Studies, Stanford University.

**Frank and Bernice J. Greenberg Professor of Law and Mark C. Mamolen Teaching Scholar, University of Chicago Law School; Herman Phleger Visiting Professor of Law, Stanford Law School. With thanks to the editors of the Harvard Law Review for insightful comments and editing.