Selected Papers of #AoIR2023: The 24th Annual Conference of the Association of Internet Researchers Philadelphia, PA, USA / 18-21 Oct 2023 Suggested Citation (APA): Iantorno, M., Guadagnolo, D., & Petterson, A. (2023, October). Dark patterns and pedagogy: Expanding scholarship and curriculum on manipulative marketing practices. Paper presented at AoIR2023: The 24th Annual Conference of the Association of Internet Researchers. Philadelphia, PA, USA: AoIR. Retrieved from http://spir.aoir.org. DARK PATTERNS AND PEDAGOGY: EXPANDING SCHOLARSHIP AND CURRICULUM ON MANIPULATIVE MARKETING PRACTICES Mathew Iantorno University of Toronto Dan Guadagnolo University of Toronto Adrian Petterson University of Toronto This conference paper addresses gaps in scholarship and pedagogy surrounding the phenomenon of “dark patterns” in digital marketing and interface design by showcasing the findings of three ongoing curriculum-building projects. Originally coined by cognitive scientist Harry Brignull in 2010, dark patterns refer to a set of design strategies that co- opt the human-centered values advocated for in the fields of user experience (UX) design and human-computer interaction (HCI) for deceptive aims (Gray et al., 2018). Not to be confused with “bad” design, dark patterns are instances where designers intentionally apply their knowledge of human psychology and persuasive design practices to manipulate users into taking an action that is contrary to their personal interests (Gray et al., 2018). Common examples of dark patterns on the Internet include advertisements disguised as user-generated posts on a social media timeline and unfavorable default settings within an app that privilege the sharing of data over privacy. Recent scholarship on dark patterns has clustered around their identification on websites and apps. This has predominantly included HCI work dedicated to codifying individual types of dark patterns (under headings such as “nagging” and “sneaking”) and quantifying their individual consequences for users on common e-commerce or social media platforms (Bösch et al., 2016; Chromik et al., 2019; Di Geronimo et al., 2020; Gray et al., 2018; Hannak et al., 2014; Luguri & Strahilevitz, 2021; Mathur et al., 2019; Mathur et al., 2021; Voigt et al., 2021). Within game studies and media studies, attention has been paid to the implementation of dark patterns in online gambling and games-as-service arrangements—often with a focus on micro-transactions, platform economies, and policy interventions (Brock & Johnson, 2021; Fitton & Read, 2019; Goodstein, 2021; Whitson & French, 2021; Zanescu et al., 2021). The projects documented in this paper build on this existing research in two substantive ways. First, they place the existing HCI, game studies, and media studies discourses on dark patterns into conversation with pertinent scholarship drawn from the domains of business and marketing, science and technology studies, cognitive neuroscience, and disability studies to create a more holistic, inclusive, and longitudinal definition of dark patterns. Second, they implement this expanded definition of dark patterns into undergraduate and graduate course curricula as a teaching tool to help students critically engage with the media platforms they use on a daily basis, the collaborative design projects they undertake for credit, and their future roles in media creation. Each of these pedagogical projects has been institutionally approved and instructed during at least one term of study. The substance of this paper consequently focuses on describing divergent and interdisciplinary approaches towards studying dark patterns; noting successes and failures in implementing these approaches into curriculum; and mapping the future work of developing such curriculum into reusable modules and methodologies. Dark Patterns: Where Marketing Meets UX Design, carried out at the University of Toronto with funding provided by a fellowship from the Critical Digital Humanities Initiative, was a one-year initiative led by principal investigators Dr. Dan Guadagnolo and Mathew Iantorno. This project challenged the assumption that dark patterns are a novel phenomenon tethered to the digital interfaces and forms of commercial exchange of Web 2.0. By exploring historical research on post-1945 market segmentation and merchandising strategies, the project evidenced how many contemporary dark patterns in fact preexist the advent of the Internet (Bargas-Avila & Hornbaek, 2011; Case, 2017; Lauer, 2017; Meyrowitz, 2009; Schwarzkopf, 2019; Taylor & Strutton, 2010; Turow, 2011; Valvi & Fragkos, 2012). Drawing together the fields of business history and critical marketing studies, this project considered how to best communicate the connections between contemporary dark patterns in e-commerce and these historical marketing and merchandising strategies to undergraduate students enrolled at the Institute of Communication, Culture, Information and Technology. To answer this question, the team developed an open-source website that visually illustrated how these analogue strategies have been transposed into digital interfaces. This website was then deployed as a pedagogical and teaching tool for Contemporary Communication Technologies, one of the largest undergraduate courses offered within the university. The second project, Dark Patterns: Manipulative UX Design, involved parallel curriculum development at the University of Toronto’s Faculty of Information. This initiative was carried out by Mathew Iantorno under the supervision of Velian Pandeliev as part of a broader pedagogy development strategy for User Interface Design, a master’s level course within the professionally oriented User Experience Design concentration. A dark patterns module that included a lecture, online resource guide, and in-class activities was developed with two main intents. First, it introduced the broad concept of dark patterns to a cohort of UX design students, enabling critical engagement with the commercial interfaces they study and the marketing practices that shape them. Second, it synthesized science and technology studies and design justice scholarship to broaden the definition of dark patterns to include non-screen interfaces. This expanded definition included the ambiance management used in casinos to motivate compulsive gambling (Lynch et al., 2020; Schüll, 2012); the construction of hostile architecture in city planning to deter loitering and protesting (Rosenberger, 2020; Smith & Walters, 2018); and the use of anthropomorphism in the design of robotics to provoke trust and empathy (Coeckelbergh, 2012; Lacey & Caudwell, 2019). The third project, Designing for Normal and Failing Ethically, examined the interaction of dark patterns with the lived experience of disabled people. This project was carried out by Adrian Petterson in collaboration with Velian Pandeliev and focused primarily on individuals with cognitive disabilities such as anxiety, autism, depression, obsessive- compulsive disorder, and attention-deficit/hyperactivity disorder. The physical differences in brain activation patterns of those with disabilities were compared to the neural networks activated by the cognitive biases exploited by dark patterns. Significant overlap was identified, primarily in areas of the brain related to fear—such as the amygdala and anterior cingulate cortex (Cavedini et al., 2004; Hartley & Phelps, 2012; Lieberman & Eisenberger, 2009; Vassena et al., 2014)—indicating that dark patterns have a disproportionate effect on individuals with these cognitive disabilities. However, legislation and pedagogy alike rarely attend to the concept of dark patterns with disability in mind. Following the principles of inclusive design (Treviranus, 2018), the project developed pedagogy demonstrating how designing for those most marginalized by a design will not only protect these vulnerable individuals but also improve user experience for everyone. This research culminated in lectures at the University of Toronto, Sheridan College, and #a11yTO focused on educating design students and professionals on these “edge case” experiences of dark patterns. Collectively, these projects aimed to grant a greater historicity and social context to the phenomenon of dark patterns and, in doing so, introduce dark patterns as a utilizable pedagogical concept within the disciplines of communications, technology, and design. The findings of these projects will be presented through the sharing of pedagogical materials; informal and formal feedback from students and associated faculty; and planned curriculum revisions for future iterations of the described initiatives. Throughout, the phenomenon of dark patterns will be posited as a complementary addition to existing methodologies in the critical analysis of design and marketing strategies, historical research and analysis, and design justice and ethical computing. References Bösch, C., Erb, B., Kargl, F., Kopp, H., & Pfattheicher, S. (2016). Tales from the dark side: Privacy dark strategies and privacy dark patterns. Proceedings on Privacy Enhancing Technologies, 2016(4), 237–254. https://doi.org/10.1515/popets- 2016-0038 Brock, T., & Johnson, M. (2021). The gamblification of digital games. Journal of Consumer Culture, 21(1), 3–13. https://doi.org/10.1177/1469540521993904 Cavedini, P., Gorini, A., & Bellodi, L. (2006). Understanding obsessive–compulsive disorder: Focus on decision making. Neuropsychology Review, 16(1), 3–15. https://doi.org/10.1007/s11065-006-9001-y Chromik, M., Eiband, M., Völkel, S. T., & Buschek, D. (2019). Dark patterns of explainability, transparency, and user control for intelligent systems. IUI Workshops 2019. ACM Digital Library. Coeckelbergh, M. (2012). Are emotional robots deceptive? IEEE Transactions on Affective Computing, 3(4), 388–393. https://doi.org/10.1109/T-AFFC.2011.29 Di Geronimo, L., Braz, L., Fregnan, E., Palomba, F., & Bacchelli, A. (2020). UI dark patterns and where to find them: A study on mobile applications and user perception. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1–14). ACM Digital Library. https://doi.org/10.1145/3313831.3376600 Fitton, D., & Read, J. C. (2019). Creating a framework to support the critical consideration of dark design aspects in free-to-play apps. Proceedings of the 18th ACM International Conference on Interaction Design and Children (pp. 407– 418). ACM Digital Library. https://doi.org/10.1145/3311927.3323136 Goodstein, S. (2021). When the cat’s away: Techlash, loot boxes, and regulating “dark patterns” in the video game industry’s monetization strategies. University of Colorado Law Review, 92(1), 285–335. Gray, C. M., Kou, Y., Battles, B., Hoggatt, J., & Toombs, A. L. (2018). The dark (patterns) side of UX design. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (pp. 1–14). ACM Digital Library. https://doi.org/10.1145/3173574.3174108 Hannak, A., Soeller, G., Lazer, D., Mislove, A., & Wilson, C. (2014). Measuring price discrimination and steering on e-commerce web sites. Proceedings of the 2014 Conference on Internet Measurement Conference (pp. 305–318). ACM Digital Library. https://doi.org/10.1145/2663716.2663744 Hartley, C. A., & Phelps, E. A. (2012). Anxiety and decision-making. Biological psychiatry, 72(2), 113–118. https://doi.org/10.1016/j.biopsych.2011.12.027 Lacey, C., & Caudwell, C. (2019). Cuteness as a “dark pattern” in home robots. Proceedings of the 2019 14th ACM/IEEE International Conference on Human- Robot Interaction (pp. 374–381). ACM Digital Library. https://doi.org/10.1109/HRI.2019.8673274 Lauer, J. (2017). Creditworthy: A history of consumer surveillance and financial identity in America. Columbia University Press. Lieberman, M. D., & Eisenberger, N. I. (2009). Pains and pleasures of social life. Science (American Association for the Advancement of Science), 323(5916), 890–891. https://doi.org/10.1126/science.1170008 Luguri, J., & Strahilevitz, L. J. (2021). Shining a light on dark patterns. Journal of Legal Analysis, 13(1), 43–109. https://doi.org/10.1093/jla/laaa006 Lynch, E., Howes, D., & French, M. (2020). A touch of luck and a “real taste of Vegas”: A sensory ethnography of the Montreal Casino. Senses & Society, 15(2): 192– 215. http://dx.doi.org/10.1080/17458927.2020.1773641 Mathur, A., Acar, G., Friedman, M., Lucherini, E., Mayer, J., Chetty, M., & Narayanan, A. (2019). Dark patterns at scale: Findings from a crawl of 11K shopping websites. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), 1–32. https://doi.org/10.1145/3359183 Mathur, A., Mayer, J., & Kshirsagar, M. (2021). What makes a dark pattern... dark? Design attributes, normative considerations, and measurement methods. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1–18). ACM Digital Library. https://doi.org/10.1145/3411764.3445610 Meyrowitz, J. (2009). We liked to watch: Television as progenitor of the surveillance society. The ANNALS of the American Academy of Political and Social Science, 625(1), 32–48. https://doi.org/10.1177/0002716209339576 Rosenberger, R. (2020). On hostile design: Theoretical and empirical prospects. Urban Studies, 57(4), 883–893. https://doi.org/10.1177/0042098019853778 Schüll, N. D. (2012). Addiction by design: Machine gambling in Las Vegas. Princeton University Press. Schwarzkopf, S. (2019). The social embeddedness of marketing. In F. F. Wherry & I. Woodward (Eds.), The Oxford Handbook of Consumption (pp. 25–49). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780190695583.013.1 Smith, N., & Walters, P. (2018). Desire lines and defensive architecture in modern urban environments. Urban Studies, 55(13), 2980–2995. https://doi.org/10.1177/004209801773269 Taylor, D. G., & Strutton, D. (2010). Has e-marketing come of age? Modeling historical influences on post-adoption era Internet consumer behaviors. Journal of Business Research, 63(9–10), 950–956. https://doi.org/10.1016/j.jbusres.2009.01.018 Treviranus, J. (2018, March). The three dimensions of inclusive design: Part one. Medium. https://medium.com/fwd50/the-three-dimensions-of-inclusive-design- part-one-103cad1ffdc2 Turow, J. (2011). The daily you: How the new advertising industry is defining your identity and your worth. Yale University Press. Valvi, A. C., & Fragkos, K. C. (2012). Critical review of the e-loyalty literature: A purchase-centred framework. Electronic Commerce Research, 12(3), 331–378. https://doi.org/10.1007/s10660-012-9097-5 Vassena, E., Silvetti, M., Boehler, C. N., Achten, E., Fias, W., & Verguts, T. (2014). Overlapping neural systems represent cognitive effort and reward anticipation. PLoS One, 9(3), e91008–e91008. https://doi.org/10.1371/journal.pone.0091008 Voigt, C., Schlögl, S., & Groth, A. (2021). Dark patterns in online shopping: Of sneaky tricks, perceived annoyance and respective brand trust. Proceedings of the 2021 HCI in Business, Government and Organizations (pp. 143–155). Springer International Publishing. https://doi.org/10.1007/978-3-030-77750-0_10 Whitson, J., & French, M. (2021). Productive play: The shift from responsible consumption to responsible production. Journal of Consumer Culture, 21(1), 14– 33. https://doi.org/10.1177/1469540521993922 Zanescu, A., Lajeunesse, M., & French, M. (2021). Speculating on Steam: Consumption in the gamblified platform ecosystem. Journal of Consumer Culture, 21(1), 34– 51. https://doi.org/10.1177/1469540521993928