Selected Papers of #AoIR2023: The 24th Annual Conference of the Association of Internet Researchers Philadelphia, PA, USA / 18-21 Oct 2023 Suggested Citation (APA): Nemer, D., & Marks, W. (2023, October). Unraveling Disinformation: Examining the Human Infrastructure of Misinformation in Brazil through the lens of Heteromation. Paper presented at AoIR2023: The 24th Annual Conference of the Association of Internet Researchers. Philadelphia, PA, USA: AoIR. Retrieved from http://spir.aoir.org. UNRAVELING DISINFORMATION: EXAMINING THE HUMAN INFRASTRUCTURE OF MISINFORMATION IN BRAZIL THROUGH THE LENS OF HETEROMATION David Nemer University of Virginia William Marks Harvard University Introduction As the information landscape continues to evolve, the widespread dissemination of disinformation is proliferating at a rate and scale never seen in a democratic society. In recent years, major technology companies have taken much of the public blame for this reality, given their algorithms facilitate the sharing of—and sometimes even promote— falsehoods. This, however, misses a key reality; social media, search engines, and messaging services are not fully automated technologies. Rather, they are heteromated: they are reliant on participatory humans to serve their economic goals. Mis- and disinformation and selective news filtering gained attention from scholars and the general public after they contributed to social media platforms’ ideological polarization that favored Donald Trump in the 2016 US presidential election and the “Brexit” vote (Vaidhyanathan, 2018). However, the rise of Brazil’s former president, the far-right firebrand Jair Bolsonaro, owes much to WhatsApp- a messaging app that runs on a peer-to-peer architecture, where no algorithm curates content according to the characteristics or demographics of the users. Instead, a human infrastructure was assembled to create a pro-Bolsonaro environment on WhatsApp and spread misinformation to bolster his candidacy (see Nemer 2021a, 2021b, 2022). In this paper, we articulate the labor executed by the Human Infrastructure of Misinformation (HIM) as heteromated labor (see Nemer & Marks, in press). As claimed by Hamid Ekbia and Bonnie Nardi (2017), there has been a shift from technologies of automation, those which are entirely managed by machines, to those of “heteromation,” which push critical tasks to end-users as indispensable mediators. Unlike technologies of automation, those of heteromation benefit from or depend on often-unrecognized or uncompensated human labor to complete tasks. The field of disinformation studies has yet to address questions related to the relationship between disinformation and labor, identity, and morality. Thus, as claimed by Jonathan Ong and Jason Cabañes (2019), approaching disinformation as a culture of production expands the field into understanding the social conditions that entice people to this work and the creative industry practices that normalize fake news as a side gig. HIM in Brazil Given the prevalence of WhatsApp use and the intriguing way that misinformation was spread during Brazil’s 2018 presidential election, David Nemer began monitoring WhatsApp groups in March 2018 that had an average of 160 members. At the peak of the election cycle, each group was posting an average of one thousand messages per day. In August, after conducting the first thematic analysis of the data, Nemer identified three clusters of actors across the groups: the Average Brazilians, the Bolso-army, and the Influencers. Nemer found that misinformation was spread in these groups through a pyramid structure, similar to the classic two-step flow of communication model (Katz and Lazarsfeld, 1966) in which each cluster of actors occupied a level. Influencers were at the top and Average Brazilians were at the bottom (see Nemer 20221a, 2021b, 2022). Heteromation of HIM HIM sustained an information system, in this case, misinformation on WhatsApp, that functioned through the actions of heterogeneous actors- in other words, the members of such infrastructure were users that provided the heteromated labor necessary to make misinformation on WhatsApp work. Ekbia and Nardi (2017) examine heteromated systems according to their functionality and reward structure. They categorize systems in terms of who benefits from the heteromated labor relation, whether participant compensation (monetary) is offered, and whether the system produces affective rewards. Beneficiaries are social actors that reap the major benefits of heteromated labor. Participants may benefit from affective rewards. Following the authors' categorization, the heteromation system that HIM labored in can be organized as detailed in Table 1 Table 1 - Heteromation of the HIM. System Heteromated functionality Beneficiaries Participant compensation Affective rewards for participants Misinformation on WhatsApp Human Infrastructure of Misinformation Politicians Influencers: medium-high Average Brazilians: high Influencers: Content- creation; cognitive and creative labor. Bolso-army: Repetitive microtasks; communicative and organizing labor. Average Brazilians: emotional and communicative labor. Bolso-army: medium-high The heteromated labor performed by Influencers and Bolso-Army was compensated in amounts of between USD 100 and USD 250 per week (between BRL 380 and BRL 900 approximately). Such an amount can be considered "medium-high" since the Brazilian minimum monthly wage in 2018 was set in at BRL 954 and the average monthly household income per capita was BRL 1.337 (IBGE, 2019). Influencers were at the top of the HIM: they were responsible for producing and bringing new misinformation to pro- Bolsonaro groups on WhatsApp. Their cognitive labor comprised of understanding the current and complex political environment in order to promote the presidential candidate Jair Bolsonaro and/or to respond to the criticism that Bolsonaro was receiving during his campaign. Such cognitive labor led to the Influencers' creative labor which meant creating outputs – like pro-Bolsonaro memes and short videos. As for Bolso-Army, they did not engage in heteromated cognitive labor, instead, they were in charge of repetitive microtasks that followed a predefined script and/or reacted to a previous trigger. Bolso-Army performed communicative labor as they were scripted to reinforce the Influencers' content's message, as soon as new content was posted to those pro-Bolsonaro groups on WhatsApp, in order to make sure that the Ordinary Brazilians internalized their message, as well as to spread the same content to other WhatsApp groups and platforms such as Twitter and groups on Facebook. Differently than claimed by Ekbia and Nardi, that described communicative labor as the act of engaging online in return for self-validation and self-promotion, members of the Bolso- Army were not necessarily after self-validation and/or self-promotion; their main goal was to validate and promote the message contained in the Influencer's posts. The Average Brazilians, performed communicative labor as they also helped spread the misinformation posted on those pro-Bolsonaro groups to their other personal groups that were not necessarily politically oriented, such as Family, Neighborhood, and Friend groups. They often used the content posted by the Influencers as evidence for their arguments in their online debates. Average Brazilians also performed emotional labor (Hochschild, 1983), since they were expected to regulate their emotions during interactions with the Bolso-Army and other Average Brazilians. Fearing being kicked out of those WhatsApp groups, they often complied with the Bolso-Army's authority and avoided asking questions that could be interpreted as them challenging Bolsonaro's legacy. Instead, they often responded to the Influencers or Bolso-Army posts by cheering with the Brazilian flag, validating messages, or with Bolsonaro’s trademark handgun emoji (👉). That validating or positive response was perceived by Average Brazilians to have their loyalty recognized by the Bolso-Army and rewarded with their permanence in the groups. Studies on heteromated systems have focused on creating awareness around labor exploitation, compensation, invisibility, and advocating for labor rights (Bailey et al. 2018; Ekbia & Nardi 2017; Irani 2019; Sambasivan et al 2021). However, in the case of the Human Infrastructure of Misinformation, given its ethical and quasilegal practices, such awareness may not be wanted nor beneficial to its members. In many cases, we might call for unpaid heteromated labor to be compensated, or for the greater acknowledgment of the work of low-paid creators. However, when it comes to the heteromated process of the HIM, the actors involved are creating what is likely a societal net negative, calling into question the types of behaviors that should be incentivized. References Bailey, D. E., Diniz, E. H., Nardi, B. A., Leonardi, P. M., & Sholler, D. (2018). A critical approach to human helping in information systems: Heteromation in the Brazilian correspondent banking system. Information and Organization, 28(3), 111-128. Ekbia, H. R., & Nardi, B. A. (2017). Heteromation, and other stories of computing and capitalism. MIT Press. Hochschild, A. R. (1983). The managed heart: Commercialization of human feeling. University of California Press IBGE. (2019, October 16). PNAD Contínua 2018: 10% da população concentram 43,1% da massa de rendimentos do país. Agência IBGE Notícias. Retrieved from https://agenciadenoticias.ibge.gov.br/agencia-sala-de-imprensa/2013-agencia-de- noticias/releases/25700-pnad-continua-2018-10-da-populacao-concentram-43-1-da- massa-de-rendimentos-do-pais Irani, L. (2019). Justice for data janitors. In Think in Public (pp. 23-40). Columbia University Press. Katz, E., & Lazarsfeld. P. F. (1966). Personal Influence, the Part Played by People in the Flow of Mass Communications. A Report of the Bureau of Applied Social Research. Columbia University Free Press. https://agenciadenoticias.ibge.gov.br/agencia-sala-de-imprensa/2013-agencia-de-noticias/releases/25700-pnad-continua-2018-10-da-populacao-concentram-43-1-da-massa-de-rendimentos-do-pais https://agenciadenoticias.ibge.gov.br/agencia-sala-de-imprensa/2013-agencia-de-noticias/releases/25700-pnad-continua-2018-10-da-populacao-concentram-43-1-da-massa-de-rendimentos-do-pais https://agenciadenoticias.ibge.gov.br/agencia-sala-de-imprensa/2013-agencia-de-noticias/releases/25700-pnad-continua-2018-10-da-populacao-concentram-43-1-da-massa-de-rendimentos-do-pais Nemer, D. (2021a). The human infrastructure of fake news in Brazil. Items. Social Sciences Research Council. Nemer, D. (2021b). Disentangling Brazil’s Disinformation Insurgency. NACLA Report on the Americas, 53(4), 406-413. Nemer, D. (2022). Technology of the Oppressed: Inequity and the Digital Mundane in Favelas of Brazil. MIT Press. Nemer, D., & Marks, W. (In Press). The Human Infrastructure of Misinformation: A Case Study of Brazil’s Heteromated Labor. In M. Ocepek, & M. Sanfilippo (Eds.), Governing Misinformation in Knowledge Commons. Cambridge University Press. Ong, J., & Cabañes, J. (2019). When Disinformation Studies Meets Production Studies: Social Identities and Moral Justifications in the Political Trolling Industry. International Journal Of Communication, 13, 20. Retrieved from https://ijoc.org/index.php/ijoc/article/view/11417/2879 Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021, May). “Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI. In proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1-15). Vaidhyanathan, S. (2018). Antisocial Media: How Facebook disconnects us and undermines democracy. Oxford University Press. https://ijoc.org/index.php/ijoc/article/view/11417/2879