Selected Papers of #AoIR2022: The 23rd Annual Conference of the Association of Internet Researchers Dublin, Ireland / 2-5 Nov 2022 Suggested Citation (APA): Ging, D., Brace, L. and Baele, S. (2022, November). Can Toxic Masculinities be De-radicalised? Mapping the Dynamics and Spread of Incel Ideology Online. Paper presented at AoIR 2022: The 23rd Annual Conference of the Association of Internet Researchers. Dublin, Ireland: AoIR. Retrieved from http://spir.aoir.org. CAN TOXIC MASCULINITIES BE DE-RADICALISED?: MAPPING THE DYNAMICS AND SPREAD OF INCEL IDEOLOGY ONLINE Debbie Ging Dublin City University Lewys Brace University of Exeter Stephane Baele University of Exeter In recent years, male supremacist and anti-women formations have become increasingly prevalent online. In particular, considerable attention has been focused on the incel (involuntary celibate) community due to a number of high-profile mass killings in the United States, Canada and, more recently, the UK. Incel ideology is a misogynistic formation, whose male proponents blame women for their lack of sexual activity. It operates in the broader virtual space of the manosphere, a loose conglomerate of online communities spread across various digital platforms, which are united in their antipathy toward feminism, their belief in evolutionary psychology and their adherence to the Red Pill (a process of enlightenment, whereby one comes to understand the world as a liberal feminist conspiracy that disadvantages men). This research tracks the dynamic pathways by which incel ideology spreads within and across online communities, digital platforms and geographical spaces, with a view to better understanding processes of radicalization, including ‘algorithmic radicalization.’ We also explore the dynamic interplay between incel and alt-right rhetoric, a phenomenon that is increasingly being noted in the academic and journalistic literature (Evans 2018; Munn 2019; Baele et al. 2020; Johanssen 2022). Understanding the contagion dynamics of extremist ideas is crucial to researchers, educators, platforms and security practitioners: how such ideas circulate, gain new audiences, morph into new ones, etc. However, theoretical and practical understanding of the online contagion of extremist ideologies is lacking. It is only by understanding these ‘pilling pipelines’ (Ging and Murphy 2021) that effective interventions can be developed, whether educational, technological, legal or platform-governance-related. The recent, evolving and growing constitution of incel ideology through repeated online interactions and offline acts of violence offers a useful case for the study of the contagion dynamics involved in online extremism. By studying the incel online space, this project systematically investigates four different axes of contagion that are at play: 1. First, incel ideology has exhibited – and continues to do so – contagion across different digital platforms, constituting a multifaceted, ever-evolving online ecosystem where activity and interactions frequently migrate from some platforms to others (which we refer to here as the “Incelosphere”). Research into the incel subculture has thus far only focused on either specific platforms (such as Reddit [Farrell et al. 2019], 4chan [Nagle 2015], YouTube [Papadamou et al. 2020], and dedicated Incel forums such as incels.co and incels.me [Baele et al. 2019; Jaki et al. 2019]) or/and on what Ging (2019) calls “flashpoint events” (such as “#gamergate” or “The Fappening”, see Massanari 2017). 2. Discussions amongst the community appear to have become increasingly political and extremist, which is indicative of some form of behavioural dynamics in relation to the internal ideological contagion of the most extreme ideas. Incel discussions have recently been shown to score similar levels of “toxicity of discussion” as well- known far-right platforms like Gab (Ribeiro et al. 2020). 3. It has also incorporated concepts seen before in other, particularly far-right, extremist ideologies, which is indicative of ideological contagion between different extremist belief systems. In turn, the growth of the incel community has meant that particular ideas, jargon and attitudes have permeated to adjacent extremist movements. As Kimmel’s (2018) work has for instance shown, the far right and other extremist groups are now adept at exploiting male anger and ‘aggrieved entitlement’ to further their own causes. Such cross-pollination of ideologies remains, however, insufficiently studied. 4. The incel movement seems to have also spread to other countries outside of North America and Canada (Witt, 2020), demonstrating geographical contagion of a scale still unknown. We use both interpretive examination of textual and image content and advanced computational methods to map the incelosphere and to track its dynamics of contagion along four key axes: 1. Radical contagion: the dynamics through which the most extreme ideas gain (or lose) traction within the Incel subculture. 2. Platform contagion: the ways in which Incel ideology spreads across different digital platforms. 3. Ideological contagion: the pathways by which Incel subcultural practices have contributed to - but also drawn from - other extremist ideologies to create a ‘cross- pollination’ of ideas, chiefly with aspects of the online far-right. 4. Geographical contagion: the geographical spread of the Incel movement, with the aim of evaluating its prominence in the UK and the Republic of Ireland. The aim of this research is to understand more precisely how the four main contagion processes listed above define the evolution of an online movement which has been linked to several incidences of gender-based terrorism over the last decade. While the majority of research conducted to date has focused on the analysis of static datasets captured from a singular platform, this study tracks the movement and transfiguration of incel ideology across several pivotal pathways. Such a detailed and dynamic examination enables us to better understand how and why young men become radicalised into this ideology, how its ideas are communicated internally and externally (linguistically and through images, e.g. memes), and how they spread and manifest across other platforms, groups and geographical spaces. Ultimately, it is hoped that such knowledge can be used to help de-radicalize disenfranchised and socially isolated men from involvement in these digital communities. References Baele, S., Brace, L. & Coan, T. (2019) ‘From “Incel” to “Saint”: Analyzing the violent worldview behind the 2018 Toronto attack’ Terrorism and Political Violence DOI: 10.1080/09546553.2019.1638256 Evans, R. (2018). From Memes to Infowars: How 75 Fascist Activists Were Red- Pilled. Bellingcat, October. Farrell, T., Fernandez, M., Novotny, J. & Alani, H. (2019) ‘Exploring Misogyny Across the Manosphere in Reddit’ WebSci 19: Proceedings of the 10th ACM Conference on Web Science 87-96. Ging, D. (2019) ‘Alphas, Betas, and Incels: Theorizing the Masculinities of the Manosphere’ Men and Masculinities 22(4), 638-657. Ging, D., & Murphy, S. (2021). Tracking the pilling pipeline: Limitations, challenges and a call for new methodological frameworks in incel and manosphere RESEARCH. AoIR Selected Papers of Internet Research. Jaki, S., De Smedt, T., Gwóźdź, M., Panchal, R., Rossa, A. & De Pauw, G. (2019) ‘Online Hatred of Women in the Incels.me Forum: Linguistic Analysis and Automatic Detection’. Journal of Language Aggression and Conflict https://doi.org/10.1075/jlac.00026.jak Johanssen, J. (2021, forthcoming) Fantasy, Online Misogyny and the Manosphere: Male Bodies of Dis/Inhibition. Routledge. Kimmel, M. (2018) Healing from Hate : How Young Men Get Into - and Out of - Violent Extremism. University of California Press. Massanari, A. (2017) ‘#Gamergate and The Fappening: How Reddit’s algorithm, governance, and culture support toxic technocultures’ New Media & Society 19,(3), 329–346. Munn, L. (2019). Alt-right pipeline: Individual journeys to extremism online. First Monday. Nagle, A. (2015) ‘Contemporary Online Anti-feminist Movements’ PhD thesis, Dublin City University, Dublin [Available online: http://doras.dcu.ie/22385/3/Angela_Nagle_PhD_thesis.pdf] Papadamou, K., Zannettou, S., Blackburn, J., De Cristofaro, E., Stringhini, G. & Sirivianos, M. (2020) ‘Understanding the Incel Community on YouTube’ Arxiv [available online: https://arxiv.org/pdf/2001.08293v1.pdf] accessed 22/03/2021 Ribeiro, M., Blackburn, J., Bradlyn, B., De Cristofaro, E., Stringhini, G., Long, S., Greenburg, S. & Zannettou, S. (2020) Arxiv [available via https://arxiv.org/abs/2001.07600]. Witt, T. (2020) ‘“If i cannot have it, i will do everything i can to destroy it.” The Canonization of Elliot Rodger: ‘Incel’ Masculinities, Secular Sainthood, and Justifications of Ideological Violence Social identities 26(5), 675-689.