Selected Papers of #AoIR2023: The 24th Annual Conference of the Association of Internet Researchers Philadelphia, PA, USA / 18-21 Oct 2023 Suggested Citation (APA): Monea, A. (2023, October). Cruising TikTok: Using Algorithmic Folk Knowledge to Evade Cisheteronormative Content Moderation. Paper (or panel) presented at AoIR2023: The 24th Annual Conference of the Association of Internet Researchers. Philadelphia, PA, USA: AoIR. Retrieved from http://spir.aoir.org. CRUISING TIKTOK; USING ALGORITHMIC FOLK KNOWLEDGE TO EVADE CISHETERONORMATIVE CONTENT MODERATION Alexander Monea George Mason University This paper examines crackdowns on queer content on TikTok and creative responses of content creators to circumvent biased content moderation and cisheteronormative censorship. The first portion of the paper demonstrates TikTok’s recurrent cisheteronormative biases in content moderation decisions and examines select instances of LGBTQ+ content that has been censored on the platform. It also works to situate this within a broader trend of LGBTQ+ censorship across internet platforms. The second portion of the paper examines how LGBTQ+ TikTok users have built up folk knowledges and intuitive understandings of TikTok’s blackboxed algorithms and opaque content moderation policies, situating this discussion within theories of the ‘algorithmic imaginary’. It catalogs the myriad ways that TikTok users work to circumvent LGBTQ+ censorship on the platform (e.g. by tactically obscuring key words in both speech and text and obscuring body parts and scenes). In the final portion of the paper, I draw on the concept of ‘cruising’ and other constitutive silences of LGBTQ+ existence to show how LGBTQ+ users are particularly well suited to producing folk knowledge about blackboxed algorithms. In closing, I examine the affordances and the limitations of LGBTQ+ users’ approach to navigating platform governance – and content moderation practices more specifically – as well as call for more organized and collective action in search of more permanent changes towards LGBTQ+-friendly platforms. TikTok has a troubled history of censoring LGBTQ+ content on its platform. It has censored depictions of homosexuality (e.g., holding hands, touching, kissing), reporting on homosexual groups, content promoting gay rights, and content promoting queerness in general in a number of conservative countries – Turkey being the most famous example (Hern 2019). Research has shown LGBTQ+ related hashtags being suppressed in at least eight languages, including Russian and Arabic (Li 2020; Ryan, Fritz & Impiombato 2020). Many American and Anglophone LGBTQ+ content creators have similarly reported biased moderation of their content, most notably amongst transgender content creators (Akinrinade 2021; Criddle 2020). TikTok has argued that many of these instances were due to their restriction of hashtags associated with ‘pornographic searches,’ but this simply reifies the pornographication and/or hypersexualization of queer existence so frequently at the center of cisheteronormative censorship. It is worth noting that TikTok’s biased content moderation extends to many other socially marginalized groups, including Black content creators (Ghaffary 2021; Rosenblatt 2021), as well as the ‘ugly,’ poor, and disabled (Biddle, Ribeiro & Dias 2020). As such, TikTok follows in the wake of many other internet platforms in its institution of cisheternormative content moderation policies (Blunt & Stardust 2021; Gehl, Moyer-Horner, & Yeo 2017; Monea 2022). Within this context, TikTok users are rightfully concerned with the undue censorship of their content and invest their time and energy into avoiding having their content censored, deprioritized, demonetized, or otherwise shadow banned. TikTok has made big promises about opening up its algorithms, policies, and decision-making practices for outside review (Heilweil 2020; Knutson 2020; Matsakis 2020), and it leverages these promises to signal its commitment to ‘accountability’ and ‘transparency’ in attempts to mitigate public relations crises – like it coming to light that TikTok censors LGBTQ+ hashtags in eight or more languages (Li 2020). However, TikTok has not actually been very forthcoming with technical details about its algorithms or data about content moderation decisions and everyday users are left with little official or expert knowledge about how TikTok’s algorithms are working and what will and will not trigger undue censorship during the content moderation process. As Abidin notes, “TikTokers have had to rely on repeated attempts, observed patterns, and gut feelings to figure out how the algorithm works, how to please the platform to facilitate their visibility, and how to have their popularity grow” (2020, p. 85). This process is akin to what Bucher calls an ‘algorithmic imaginary,’ which describes “the way in which people imagine, perceive and experience algorithms and what these imaginations make possible” (2017, p. 31), as well as Bishop’s work on ‘algorithmic gossip’ (2019) and ‘algorithmic lore’ (2020). Others have described this process as ‘algorithmic folklore,’ writing: TikTok users have speculated about coded discrimination on the platform, sharing individual experiences and anecdotal evidence to identify and disrupt the algorithm. Engaging in collective guesswork, these users take to the comments section to propose different theories about why the algorithm acts in discriminatory ways. […] By sharing experiences, asking questions, and crowdsourcing answers, teens are developing an algorithmic folklore while discerning the potential motivations behind TikTok’s software engineering. (Akinrinade 2021) This is a departure from traditional folk psychology, which explains the ways in which humans perceive, explain, predict, and criticize one another’s behavior – largely through the attribution of mental states to others (Hutto & Ravenscroft 2021). LGBTQ+ users are particularly adept at producing folk knowledge about TikTok’s algorithms and content moderation policies, given both the general cisheteronormative bias of online content moderation and TikTok’s specific history of censoring LGBTQ+ content on its platform. The practices they employ to identify and avoid cisheteronormative content moderation practices on the platform include (but are not limited to): • Intentionally using language, keywords, hashtags, and images that they anticipate triggering censorship and cataloging TikTok’s responses • Continually A/B testing the platform’s content moderation by posting multiple iterations of the same videos with slight alterations to see which elude the algorithm’s content moderation • Engaging in motivating misspelling and mispronunciation of words understood to trigger censorship (e.g. ‘seggs’ or ‘s*ggs’ in place of ‘sex’) • Tactically covering or obscuring certain parts of the body, background objects, portions of images, etc. • Leveraging specific audiences, hashtags, and cross platform links to boost LGBTQ+ content • Utilizing comment spaces and forums on other platforms to collectively produce, collect, archive, and disseminate folk knowledge for LGBTQ+ TikTok users In fact, LGBTQ+ content creators might be particularly well positioned to produce these folk knowledges for evading cisheteronormative algorithmic censorship. As Sedgwick (2008) has shown, queer folx are particularly well versed in navigating the constitutive silences of discourse, determining safe and unsafe conditions of speech and the attendant meanings of such determinations. This capacity is paralleled by the ability to navigate visibility and invisibility in public spaces, particularly as evidenced by ‘cruising’ wherein queer folx navigate precarious but coded acts of public disclosure of their sexuality in search of erotic partners (Bullock 2004; Humphreys 1970; Leap 1999). Algorithmic folk knowledges require similar capacities to navigate speech and silence, visibility and invisibility, all with increasingly high stakes for participating in everyday digitally connected life. In closing, I suggest that perhaps we might follow Muñoz (2009) in looking to these practices of ‘algorithmic cruising’ for a more utopic digital future. References Abidin, C. (2020). Mapping Internet Celebrity on TikTok: Exploring Attention Economies and Visibility Labours. Cultural Science, 12(1): 77-103. Akinrinade, I. (2021, Jul. 14). Strategic Knowledge: Teens Use “Algorithmic Folklore” to Crack TikTok’s Black Box. Data & Society. Retrieved from https://points.datasociety.net/strategic-knowledge-6bbddb3f0259 [Accessed Mar. 1, 2023] Biddle, S., Ribeiro, P.V., Dias, T. (2020, Mar. 16). Invisible Censorship: TikTok Told Moderators to Suppress Posts by “Ugly” People and the Poor to Attract New Users. The Intercept. Retrieved from https://theintercept.com/2020/03/16/tiktok-app-moderators- users-discrimination/ [Accessed Mar. 1, 2023] Bishop, S. (2019). Managing Visibility on YouTube through Algorithmic Gossip. New Media & Society, 21(11-12): 2589-2606. https://points.datasociety.net/strategic-knowledge-6bbddb3f0259 https://theintercept.com/2020/03/16/tiktok-app-moderators-users-discrimination/ https://theintercept.com/2020/03/16/tiktok-app-moderators-users-discrimination/ Bishop, S. (2020). Algorithmic Experts: Selling Algorithmic Lore on YouTube. Social Media + Society, 6(1): 1-11. Blunt, D., & Stardust, Z. (2021). Automating Whorephobia: Sex, Technology and the Violence of Deplatforming: An Interview with Hacking//Hustling. Porn Studies, 8(4), 350- 366. Bucher, T. (2017). The Algorithmic Imaginary: Exploring the Ordinary Affects of Facebook Algorithms. Information, Communication & Society, 20(1): 30-44. Bullock, D. (2004). Lesbian Cruising. Journal of Homosexuality, 47(2): 1-31. Criddle, C. (2020, Feb. 12). Transgender Users Accuse TikTok of Censorship. BBC News. Retrieved from https://www.bbc.com/news/technology-51474114 [Accessed Mar. 1, 2023] Gehl, R. W., Moyer-Horner, L., & Yeo, S. K. (2017). Training Computers to See Internet Pornography: Gender and Sexual Discrimination in Computer Vision Science. Television & New Media, 18(6), 529-547. Ghaffary, S. (2021, Jul. 7). How TikTok’s Hate Speech Detection Tool Set Off a Debate About Racial Bias on the App. Vox. Retrieved from https://www.vox.com/recode/2021/7/7/22566017/tiktok-black-creators-ziggi-tyler-debate- about-black-lives-matter-racial-bias-social-media [Accessed Mar. 1, 2023] Heilweil, R. (2020, Jul. 29). Why it Matters that TikTok Wants to Reveal its Algorithms. Vox. Retrieved from https://www.vox.com/recode/2020/7/29/21346758/tiktok-for-you- algorithm-transparency-instagram-antitrust [Accessed Mar. 1, 2023] Hern, A. (2019, Sep. 26). TikTok’s Local Moderation Guidelines Ban Pro-LGBT Content. The Guardian. Retrieved from https://www.theguardian.com/technology/2019/sep/26/tiktoks-local-moderation- guidelines-ban-pro-lgbt-content [Accessed Mar. 1, 2023] Humphreys, L. (1970). Tearoom trade: Impersonal sex in public places. Chicago, IL: Aldine. Hutto, D. & Ravenscroft, I. (2021). Folk Psychology as a Theory. Stanford Encyclopedia of Philosophy. Retrieved from https://plato.stanford.edu/entries/folkpsych-theory/ [Accessed Mar. 1, 2023] Knutson, J. (2020, Mar. 11). TikTok Plans Los Angeles “Transparency Center” to Assuage Critics. Axios. Retrieved from https://www.axios.com/2020/03/11/tiktok-los- angeles-china [Accessed Mar. 1, 2023] Leap, W. L. (Ed.). (1999). Public sex/Gay space. New York, NY: Columbia University Press. https://www.bbc.com/news/technology-51474114 https://www.vox.com/recode/2021/7/7/22566017/tiktok-black-creators-ziggi-tyler-debate-about-black-lives-matter-racial-bias-social-media https://www.vox.com/recode/2021/7/7/22566017/tiktok-black-creators-ziggi-tyler-debate-about-black-lives-matter-racial-bias-social-media https://www.vox.com/recode/2020/7/29/21346758/tiktok-for-you-algorithm-transparency-instagram-antitrust https://www.vox.com/recode/2020/7/29/21346758/tiktok-for-you-algorithm-transparency-instagram-antitrust https://www.theguardian.com/technology/2019/sep/26/tiktoks-local-moderation-guidelines-ban-pro-lgbt-content https://www.theguardian.com/technology/2019/sep/26/tiktoks-local-moderation-guidelines-ban-pro-lgbt-content https://plato.stanford.edu/entries/folkpsych-theory/ https://www.axios.com/2020/03/11/tiktok-los-angeles-china https://www.axios.com/2020/03/11/tiktok-los-angeles-china Li, J. (2020, Sep. 8). TikTok is Suppressing LGBT Content in Eastern Europe and the Middle East. Quartz. Retrieved from https://qz.com/1900530/tiktok-shadow-bans-lgbt- hashtags-in-russian-and-arabic [Accessed Mar. 1, 2023] Matsakis, L. (2020, Jun. 18). TikTok Finally Explains How the ‘For You’ Algorithm Works. WIRED. Retrieved from https://www.wired.com/story/tiktok-finally-explains-for- you-algorithm-works/ [Accessed Mar. 1, 2023] Muñoz, J. E. (2009). Cruising Utopia: The Then and There of Queer Futurity. New York, NY: NYU Press. Rosenblatt, K. (2021, Feb. 9). Months after TikTok Apologized to Black Creators, Many Say Little Has Changed. NBC News. Retrieved from https://www.nbcnews.com/pop- culture/pop-culture-news/months-after-tiktok-apologized-black-creators-many-say-little- has-n1256726 [Accessed Mar. 1, 2023] Ryan, F., Fritz, A., & Impiombato, D. (2020). TikTok and WeChat: Curating and Controlling Global Information Flows. Policy Brief Report No. 37/2020. Canberra, Australia: Australian Strategic Policy Institute. Retrieved from https://www.aspi.org.au/report/tiktok-wechat [Accessed Mar. 1, 2023] Sedgwick, E. K. (2008). Epistemology of the closet. Berkeley and Los Angeles, CA: University of California Press. https://qz.com/1900530/tiktok-shadow-bans-lgbt-hashtags-in-russian-and-arabic https://qz.com/1900530/tiktok-shadow-bans-lgbt-hashtags-in-russian-and-arabic https://www.wired.com/story/tiktok-finally-explains-for-you-algorithm-works/ https://www.wired.com/story/tiktok-finally-explains-for-you-algorithm-works/ https://www.nbcnews.com/pop-culture/pop-culture-news/months-after-tiktok-apologized-black-creators-many-say-little-has-n1256726 https://www.nbcnews.com/pop-culture/pop-culture-news/months-after-tiktok-apologized-black-creators-many-say-little-has-n1256726 https://www.nbcnews.com/pop-culture/pop-culture-news/months-after-tiktok-apologized-black-creators-many-say-little-has-n1256726 https://www.aspi.org.au/report/tiktok-wechat