Selected Papers of #AoIR2024: The 25th Annual Conference of the Association of Internet Researchers Sheffield, UK / 30 Oct - 2 Nov 2024 Suggested Citation (APA): Frizzera, L. (2024, October). Predictions of the Self: AI and the Political Economy of Subjectivation. Paper presented at AoIR2024: The 25th Annual Conference of the Association of Internet Researchers. Sheffield, UK: AoIR. Retrieved from http://spir.aoir.org. PREDICTIONS OF THE SELF: AI AND THE POLITICAL ECONOMY OF SUBJECTIVATION Luciano Frizzera Concordia University The recent widespread availability of Artificial Intelligence (AI) technology and the extensive records of human activities and behaviour in digital format present serious challenges related to how individuals construct their own identities and social relations. AI systems datafy our body—our face, our gender, the colour of our skin—and our identity, producing live databases for computable linkages between humans and nonhumans. They create a new cartography of biopower (Foucault, 1982), producing a new form of political economy of subjectivation that treats individuals as objects from which raw material is extracted to produce predictive models that act as our data doubles (Haggerty & Ericson, 2000). Focusing on Kaggle, a platform for crowdsourcing AI development, I examine the practices of the data science community on three high- profile machine learning projects and conclude by arguing that machine learning has been thought of and developed as a prediction of the self in order to prescribe individual behaviour to fulfill specific economic conditions. Subjectivation (Foucault, 2017) is a central concept for the constitution of individuals, whereby one’s sense of self as an individual agent is paradoxically shaped according to forces external to the self. Subjectivation has been described as the totalizing power of the state (Althusser, 1971), decentralized and woven through everyday life (Foucault, 2017), and embodied experience (Butler, 1990; Browne, 2015). With the technological transformations over the last three decades, subjectivation has become impersonal and algorithmically driven (Chun, 2012; Langlois & Elmer, 2019). AI complicates things as it automatizes subjectivation using predictive models. These models are used to identify, sort, and classify populations; to shape individual behaviour and habits; to create value and capital; to modulate our experience of and within the world. AI systems interpellate individuals as subjects of the digital society, not only to reinforce our conditions of existence (Lazzarato, 2004) but also to (re)mediate our embodied experiences (Grusin, 2015). As such, we must think of AI as a discourse that produces specialized knowledge based on specific inputs from datasets as a way to manage populations. AI takes on the historical forces of capitalism, colonialism, patriarchy, and racism and disseminates and solidifies these logics in societies, asymmetrically influencing social groups. It emerges as an instrument of technocapitalism, which can only further the agenda of the systems in which it is embedded. Issues such as algorithmic social and cultural biases (Apprich et al., 2018; Bolukbasi et al., 2016; Xu et al., 2022), the idealized and pragmatic economic uses of AI (Finn, 2017; Hong, 2020; Srnicek, 2017), and the consequent reproduction of already existing power structures by predictive models (Amoore, 2020; Crawford, 2021; O’Neil, 2016) have been problematized in the literature. However, most of these contributions focus on specific algorithms or the use of predictive models by large digital infrastructures at the consumer level. This paper asks what kinds of data and labour mobilization occur in and around the production of predictive models: What political economy and socio-technical conditions are involved in the production of AI? How do these conditions produce predictive models that shape our sense of self and identity? For that, we must focus on places where AI systems are iteratively prototyped, tested, and refined, tasks often run by volunteers and low-wage workers, such as Kaggle, a crowdsourcing platform for data science and machine learning development based on competitive events. With over 10 million users, Kaggle has held hundreds of public large-scale machine learning competitions over the last 14 years. These competitions have cash prizes of up to US$ 3 million, attracting thousands of highly trained computer scientists, engineers, and data scientists worldwide. Most of these competitions are sponsored by large private companies (e.g., Facebook, Google, Walmart) and public agencies (e.g., US Department of Homeland Security) aiming to solve problems related to optimization and human behaviour, most notably for commercial or security purposes. I argue that Kaggle is a site of impersonal subjectivation where value is mobilized through large assemblages of data, code, crowdsourced labour, and specific economic interests. This paper focuses on three high-profile competitions: Deepfake Detection Challenge, Passenger Screening Algorithm Challenge, and Instacart Market Basket Analysis. These competitions exhibit ostensibly untroubled data mobilization for algorithmic subjectivation. Each of these competitions had a specific goal that matches a topology for producing socio-techno apparatuses of control: identify, predict, and mediate. Drawing on Digital Methods (Rogers, 2009) and Software Studies (Fuller, 2008), I examine how these competitions unfold, what algorithms developers used, what kinds of predictive models were produced, and the user’s interactions in the website’s forum. Participants in these competitions exchanged over 11 thousand messages in the forum and produced dozens of thousands of predictive models. I analyzed participants’ contributions to describe how machine learning has been thought of and developed as a prediction of the self in order to prescribe individual behaviour to fulfill specific economic conditions. I conclude by drawing on critical theory and discourse analysis to argue that more than just a way of doing computation, Kaggle competitions show that machine learning development turned into a dogmatic capitalist mode of production with little care for human life. It pervades and regulates societies at a political, social and cultural level. This phenomenon, which I call modes of automatic subjectivation, points toward the possibility of using subjective and impersonal materials to reorganize life in its broadest sense according to a specific system of power and privileges involving gender, race, sexuality, and social class. Crucially, I argue that these modes of subjectivation are designed to control the “production of possibilities” and reinforce specific types of socioeconomic relations, creating the conditions of existence that determine how resources and people are organized. References Althusser, L. (1971). Ideology and Ideological State Apparatuses. In Lenin and Philosophy (pp. 123–156). New Left Books. Amoore, L. (2020). Cloud Ethics: Algorithms and the Attributes of Ourselves and Others. Duke University Press. Apprich, C., Chun, W. H. K., Cramer, F., & Steyerl, H. (2018). Pattern Discrimination. Meson Press. Bolukbasi, T., Chang, K.-W., Zou, J. Y., Saligrama, V., & Kalai, A. T. (2016). Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. 30th Conference on Neural Information Processing Systems (NIPS 2016), 9. Browne, S. (2015). Dark matters: On the surveillance of blackness. Duke University Press. Butler, J. (1990). Gender Trouble. Routledge. Chun, W. H. K. (2017). Updating to Remain the Same: Habitual New Media (Reprint edition). The MIT Press. Crawford, K. (2021). Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. Yale University Press. Finn, E. (2017). What Algorithms Want: Imagination in the Age of Computing. The MIT Press. Foucault, M. (1982). The Subject and Power. Critical Inquiry, 8(4), 777–795. https://doi.org/10.1086/448181 Foucault, M. (2017). Subjectivity and Truth: Lectures at the Collège de France, 1980- 1981 (F. Ewald, A. Fontana, & F. Gros, Eds.; G. Burchell, Trans.; 1st ed. 2017 edition). Palgrave Macmillan. Fuller, M. (Ed.). (2008). Software Studies: A Lexicon. MIT Press. Grusin, R. (2015). Radical Mediation. Critical Inquiry, 42(1), 124–148. https://doi.org/10.1086/682998 Haggerty, K. D., & Ericson, R. V. (2000). The surveillant assemblage. The British Journal of Sociology, 51(4), 605–622. https://doi.org/10.1080/00071310020015280 https://doi.org/10.1080/00071310020015280 Hong, S. (2020). Technologies of Speculation: The Limits of Knowledge in a Data- Driven Society. NYU Press. Langlois, G., & Elmer, G. (2019). Impersonal subjectivation from platforms to infrastructures. Media, Culture & Society, 41(2), 236–251. https://doi.org/10.1177/0163443718818374 Lazzarato, M. (2004). From Capital-Labour to Capital-Life (V. Fournier, A. Virtanen, & J. Vâhâmâki, Trans.). Ephemera, 4(3), 187–208. O’Neil, C. (2016). Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy (Reprint edition). Broadway Books. Rogers, R. (2009). The End of the Virtual: Digital Methods. Amsterdam University Press. https://doi.org/10.5117/9789056295936 Srnicek, N. (2017). Platform Capitalism. Polity. Xu, Y., Terhörst, P., Raja, K., & Pedersen, M. (2022). A Comprehensive Analysis of AI Biases in DeepFake Detection With Massively Annotated Databases (arXiv:2208.05845).