Selected Papers of #AoIR2024: The 25th Annual Conference of the Association of Internet Researchers Sheffield, UK / 30 Oct - 2 Nov 2024 Suggested Citation (APA): Cotter, K. (2024, October). Epistemic-Democratic Tension in The Bottom-Up Governance of Algorithms. Paper presented at AoIR2024: The 25th Annual Conference of the Association of Internet Researchers. Sheffield, UK: AoIR. Retrieved from http://spir.aoir.org. EPISTEMIC-DEMOCRATIC TENSION IN THE BOTTOM-UP GOVERNANCE OF ALGORITHMS Kelley Cotter The Pennsylvania State University Introduction Threats posed by platform algorithms to privacy, agency, fairness, and equity, particularly the reenactment and mediation of systems of power (Benjamin, 2019; Noble, 2018; O’Neil, 2016), underscore the imperative for robust governance measures. Discussions on algorithmic governance often center on top-down approaches developed primarily by policymakers and private companies (Bloch-Wehba, 2022; Saurwein et al., 2015). While these are crucial, the involvement of those most affected by algorithms—everyday people—has received less attention. As governments seek the input of their citizens on relevant concerns—for example, listening sessions to inform a blueprint for the AI Bill of Rights in the U.S. (White House Office of Science and Technology Policy, 2022) and The European AI Alliance’s online forum for discussions (European Commission, 2024)—it is imperative to define the parameters of meaningful participation by “ordinary” individuals in the governance of algorithms. In this paper, I explore the concept of "bottom-up governance" of platform algorithms, emphasizing the "epistemic-democratic tension" between the "participation of the affected" and "expertise-based decision-making." I argue that bottom-up governance requires the involvement of "lay" experts to ensure algorithms function fairly. The realization of this ideal is complex, as what people know about algorithms represents a social and political negotiation of meaning. To be heard in public deliberation about algorithms, we need to expand the horizon of what counts as legitimate and consequential knowledge. In what follows, I begin by offering a broad vision what it means to know algorithms based on a synthesis of the cross-disciplinary scholarship in this area. I then connect this understanding of algorithmic expertise with understandings of participatory democracy to offer two key principles to guide the realization of bottom- up governance. These are: alertness to the authority of non-technical knowers and subjugated knowledges. Knowing Algorithms The theoretical vocabulary explaining how everyday people understand algorithms has grown increasingly complex over the last several years, with frameworks including the algorithmic imaginary (Bucher, 2017), algorithmic folk theories (e.g., DeVito, 2021; DeVito et al., 2018; Eslami et al., 2016; Siles et al., 2020), algorithmic knowledge (Cotter, 2022; Cotter & Reisdorf, 2020); algorithmic literacy (Cotter, 2020; DeVito, 2021; Dogruel et al., 2021; Oeldorf-Hirsch & Neubaum, 2023), algorithm skills (Gruber & Hargittai, 2023; Klawitter & Hargittai, 2018), algorithmic competencies (Jarrahi & Sutherland, 2019), algorithmic gossip (Bishop, 2019), hermeneutics of algorithm (Andersen, 2020), everyday auditing (Shen et al., 2021), among others. This literature encompasses three high-level, intersecting visions of knowledge: knowledge located in thought, knowledge located in bodies (practices, affective states), and knowledge as heterogeneous and context-contingent (Cotter, 2022; Oeldorf-Hirsch & Neubaum, 2023). Although technical knowledge tends to take the spotlight in discussions of algorithmic sensemaking and understanding, other knowledges represent key mechanisms affecting how we collectively decide to govern algorithms. For one, much of people’s knowledge about algorithms is tacit, unspoken, and intuitive (Bucher, 2018; Ruckenstein & Granroth, 2020; Swart, 2021). Our affective responses and embodied insights directly guide how we orient ourselves to algorithms and how we formulate a sense of when algorithms are helpful or harmful. A narrow focus on technical insights may lead to an underestimation of what people know about algorithms and the value of their everyday wisdom. Emphasis on technical knowledge often implicates deficit models of digital literacy, where limited digital engagement and skills are presumed to result from “shortfalls in cognition, personality, knowledge, resourcing, social situation or personal ideology” (Selwyn, 2003, pp. 106–107). Everyday insights about algorithms incubate in individuals’ experiences in the world, their personalities, psychology, beliefs, and attitudes, as guided by sociocultural contexts. Given algorithms’ highly contextual, contingent nature, people’s situated encounters with these computational processes give rise to unique insight about their impact (Shen et al., 2021), particularly among disempowered groups (Andalibi & Garcia, 2021; DeVito, 2022; Duffy & Meisner, 2023; Haimson et al., 2021; Karizat et al., 2021; Simpson & Semaan, 2021). Some knowledges about algorithms are not universal and need not be. The multiplicity of knowledges about algorithms requires that modes of algorithmic governance account for the broadest range of input possible. Epistemic-Democratic Tension Public deliberation stands as a defining pillar of democracy, anchored in the vision of a public sphere where citizens engage in "unrestricted rational discussion of public matters" (Fraser, 1990, p. 59). The pursuit of "participation parity" encapsulates the ideal of the public sphere, aspiring to forge a shared space where all individuals can participate on equal terms. However, stratified societies require multiple counterpublics, where segments of the population, particularly those with less power, confront and contest dominant perspectives within the broader public discourse (Fraser, 1990). While participatory approaches to governance are considered a normative good for democracy, contention arises over the question of who should be involved. Contemporary governance trends reveal a growing reliance those deemed experts with specialized, credentialed knowledge (Krick, 2022). This understanding of expertise perpetuates knowledge hierarchies, resulting in power imbalances that disproportionately favor those deemed experts (Krick, 2022, p. 469) who tend to represent elite classes. Bottom-Up Governance If we want algorithms to function in harmony with the needs and interests of diverse publics, then we need diverse publics to be involved in conversations about algorithmic governance. Further, we need to take what members of these publics—everyday people—know seriously. Bottom-up governance extends beyond merely soliciting input from citizens on algorithms; it necessitates embracing two fundamental principles. Firstly, bottom-up governance requires accepting everyday people as authorities on what algorithms mean to and for them. Thus far, governance structures have prioritized input from elite stakeholders—researchers, industry practitioners, policymakers, journalists. These stakeholders play an essential role in studying and bringing to light the unintended consequences of algorithms. Yet, these elite stakeholders cannot see algorithms from all possible angles, which means they often miss critical problems (e.g., Shen et al., 2021). Moreover, knowledge and power remain deeply intertwined, with the potential for deficit model mindsets premised on “deficient” technical knowledge to obstruct full, effective participation in public deliberation on algorithms. A lack of vocabulary or conceptual framework to describe algorithms on a technical level does not negate the potential for meaningful understanding of their embedded values and the associated consequences. In essence, we must understand “expertise” as contextual and diverse. Secondly, bottom-up governance requires the specific expertise of members of marginalized groups to surface critical insight on the ways algorithms (re)enact political order (Bucher, 2018; Noble, 2018). Here, prioritizing “subjugated standpoints” is essential (Collins, 2014; Haraway, 1988), as they offer a unique vantage point into the mechanisms through which algorithms sustain historical patterns of systemic inequalities. Algorithms impact different individuals and communities in different ways and to different degrees. Existing work demonstrates that BIPOC, LGBTQ+ members, and other marginalized groups bear the brunt of negative impacts (e.g., DeVito, 2022; Duffy & Meisner, 2023; Haimson et al., 2021; Karizat et al., 2021). While policymakers care about the protection of their constituents, their enthusiasm for technological advancements that could benefit their national economies and enhance global influence may sometimes overshadow this primary goal (e.g., Suchman & Whittaker, 2021). Bottom-up approaches that center disempowered communities emphasizes justice and protecting human rights first and foremost. Effectively integrating "participation of the affected" and "expertise-based decision- making" demands further scholarly consideration, guiding us towards techniques that meaningfully support consultation and deliberative contestation by those directly affected by algorithms, particularly the most vulnerable. References Andalibi, N., & Garcia, P. (2021). Sensemaking and Coping After Pregnancy Loss: The Seeking and Disruption of Emotional Validation Online. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), 1–32. https://doi.org/10.1145/3449201 Andersen, J. (2020). Understanding and interpreting algorithms: Toward a hermeneutics of algorithms. Media, Culture & Society, 42(7–8), 1479–1494. https://doi.org/10.1177/0163443720919373 Benjamin, R. (2019). Race after technology: Abolitionist tools for the new Jim code. Polity. Bishop, S. (2019). Managing visibility on YouTube through algorithmic gossip. New Media & Society, 21(11–12), 2589–2606. https://doi.org/10.1177/1461444819854731 Bloch-Wehba, H. (2022). Algorithmic Governance from the Bottom Up. Brigham Young University Law Review, 48(1), 69–136. https://doi.org/10.2139/ssrn.4054640 Bucher, T. (2017). The algorithmic imaginary: Exploring the ordinary affects of Facebook algorithms. Information, Communication & Society, 20(1), 30–44. https://doi.org/10.1080/1369118X.2016.1154086 Bucher, T. (2018). If...then: Algorithmic power and politics. Oxford University Press. Collins, P. H. (2014). Black feminist thought: Knowledge, consciousness, and the politics of empowerment. Cotter, K. (2020). Critical algorithmic literacy: Power, epistemology, and platforms [Doctoral dissertation]. Michigan State University. Cotter, K. (2022). Practical knowledge of algorithms: The case of BreadTube. New Media & Society, 146144482210818. https://doi.org/10.1177/14614448221081802 Cotter, K., & Reisdorf, B. C. (2020). Algorithmic knowledge gaps: A New horizon of (digital) inequality. International Journal of Communication, 14. https://ijoc.org/index.php/ijoc/article/view/12450/2952 DeVito, M. A. (2021). Adaptive Folk Theorization as a Path to Algorithmic Literacy on Changing Platforms. Proceedings of the ACM on Human-Computer Interaction, 4(CSCW2), 1–35. https://doi.org/10.1145/3476080 DeVito, M. A. (2022). How Transfeminine TikTok Creators Navigate the Algorithmic Trap of Visibility Via Folk Theorization. Proceedings of the ACM on Human- Computer Interaction, 6(CSCW2), 1–31. https://doi.org/10.1145/3555105 DeVito, M. A., Birnholtz, J., Hancock, J. T., French, M., & Liu, S. (2018). How people form folk theories of social media feeds and what it means for how we study self- presentation. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 1–12. https://doi.org/10.1145/3173574.3173694 Dogruel, L., Masur, P., & Joeckel, S. (2021). Development and Validation of an Algorithm Literacy Scale for Internet Users. Communication Methods and Measures, 1–19. https://doi.org/10.1080/19312458.2021.1968361 Duffy, B. E., & Meisner, C. (2023). Platform governance at the margins: Social media creators’ experiences with algorithmic (in)visibility. Media, Culture & Society, 45(2), 285–304. https://doi.org/10.1177/01634437221111923 Eslami, M., Karahalios, K., Sandvig, C., Vaccaro, K., Rickman, A., Hamilton, K., & Kirlik, A. (2016). First I “like” it, then I hide it: Folk Theories of Social Feeds. 2371– 2382. https://doi.org/10.1145/2858036.2858494 European Commission. (2024, February 27). The European AI Alliance. https://digital- strategy.ec.europa.eu/en/policies/european-ai-alliance Fraser, N. (1990). Rethinking the Public Sphere: A Contribution to the Critique of Actually Existing Democracy. Social Text, 25/26, 56. https://doi.org/10.2307/466240 Gruber, J., & Hargittai, E. (2023). The importance of algorithm skills for informed Internet use. Big Data & Society, 10(1), 205395172311681. https://doi.org/10.1177/20539517231168100 Haimson, O. L., Delmonaco, D., Nie, P., & Wegner, A. (2021). Disproportionate Removals and Differing Content Moderation Experiences for Conservative, Transgender, and Black Social Media Users: Marginalization and Moderation Gray Areas. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW2), 1–35. https://doi.org/10.1145/3479610 Haraway, D. (1988). Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective. Feminist Studies, 14(3), 575. https://doi.org/10.2307/3178066 Jarrahi, M. H., & Sutherland, W. (2019). Algorithmic Management and Algorithmic Competencies: Understanding and Appropriating Algorithms in Gig Work. In N. G. Taylor, C. Christian-Lamb, M. H. Martin, & B. Nardi (Eds.), Information in Contemporary Society (pp. 578–589). Springer International Publishing. Karizat, N., Delmonaco, D., Eslami, M., & Andalibi, N. (2021). Algorithmic Folk Theories and Identity: How TikTok Users Co-Produce Knowledge of Identity and Engage in Algorithmic Resistance. Proceedings of the ACM on Human-Computer Interaction CSCW2 (2021), Forthcoming. Klawitter, E., & Hargittai, E. (2018). “It’s like learning a whole other language:” The role of algorithmic skills in the curation of creative goods. International Journal of Communication, 12, 3490–3510. Krick, E. (2022). Participatory Governance Practices at the Democracy-Knowledge- Nexus. Minerva, 60(4), 467–487. https://doi.org/10.1007/s11024-022-09470-z Noble, S. U. (2018). Algorithms of oppression: How search engines reinforce racism. New York University Press. Oeldorf-Hirsch, A., & Neubaum, G. (2023). What do we know about algorithmic literacy? The status quo and a research agenda for a growing field. New Media & Society, 14614448231182662. https://doi.org/10.1177/14614448231182662 O’Neil, C. (2016). Weapons of math destruction: How big data increases inequality and threatens democracy. Crown. Ruckenstein, M., & Granroth, J. (2020). Algorithms, advertising and the intimacy of surveillance. Journal of Cultural Economy, 13(1), 12–24. https://doi.org/10.1080/17530350.2019.1574866 Saurwein, F., Just, N., & Latzer, M. (2015). Governance of algorithms: Options and limitations. Info, 17(6), 35–49. https://doi.org/10.1108/info-05-2015-0025 Selwyn, N. (2003). Apart from technology: Understanding people’s non-use of information and communication technologies in everyday life. Technology in Society, 25(1), 99–116. https://doi.org/10.1016/S0160-791X(02)00062-3 Shen, H., DeVos, A., Eslami, M., & Holstein, K. (2021). Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. arXiv Preprint arXiv:2105.02980. Siles, I., Segura-Castillo, A., Solís, R., & Sancho, M. (2020). Folk theories of algorithmic recommendations on Spotify: Enacting data assemblages in the global South. Big Data & Society, 7(1), 1–15. https://doi.org/10.1177/2053951720923377 Simpson, E., & Semaan, B. (2021). For You, or For"You"?: Everyday LGBTQ+ Encounters with TikTok. Proceedings of the ACM on Human-Computer Interaction, 4(CSCW3), 1–34. https://doi.org/10.1145/3432951 Suchman, L., & Whittaker, M. (2021, December 8). The Myth of Artificial Intelligence. The American Prospect. https://prospect.org/api/content/7fc7f7c2-5781-11ec- 987e-12f1225286c6/ Suresh, H., & Guttag, J. (2021). A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle. Equity and Access in Algorithms, Mechanisms, and Optimization, 1–9. https://doi.org/10.1145/3465416.3483305 Swart, J. (2021). Experiencing algorithms: How young people understand, feel about, and engage with algorithmic news selection on social media. Social Media + Society, 7(2), 1–11. https://doi.org/10.1177/20563051211008828 White House Office of Science and Technology Policy. (2022). Blueprint for an AI Bill of Rights. The White House. https://www.whitehouse.gov/ostp/ai-bill-of-rights/