Selected Papers of #AoIR2024: The 25th Annual Conference of the Association of Internet Researchers Sheffield, UK / 30 Oct - 2 Nov 2024 Suggested Citation (APA): Robinson, J. Y. & Cole, S. (2024, October). Proposing reciprocal digital methods: A user-centric method for algorithmic social media platforms in a post-API world. Paper presented at AoIR2024: The 25th Annual Conference of the Association of Internet Researchers. Sheffield, UK: AoIR. Retrieved from http://spir.aoir.org. PROPOSING RECIPROCAL DIGITAL METHODS: A USER-CENTRIC METHOD FOR ALGORITHMIC SOCIAL PLATFORMS IN A POST-API WORLD Jessica Yarin Robinson University of Oslo Sebastian Cole University of Oslo Introduction This paper introduces reciprocal digital methods, a novel research framework tailored to the exigencies of studying social media in what has been called a post-API landscape (Bruns, 2019; Freelon, 2018). In this paper we build on scholarly discourse on the epistemology and ethics of social media data (Lomborg & Bechmann, 2014; Marres & Gerlitz, 2016), and the current debates about the future of social media research (Bruns, 2019; Freelon, 2018; Ohme & Araujo, 2022; Tromble, 2021). We propose a model that is intended to push the field forward, merging approaches to social media that have been largely disparate, and combining computational analysis of user-level digital trace data and interviews with the same users. The background for this method is the ascendance of algorithmic social platforms, which have engendered an ecosystem wherein user interactions are mediated and molded by opaque and highly tailored algorithmic processes. Digital data, while rich in potential insights, often lacks the context necessary to interpret user behavior and platform interaction accurately. Conversely, interviews provide depth and narrative but are generally not reliable for capturing use patterns. By combining these two elements, the proposed methodology enables researchers to bridge the gap between narrative and pattern, and between media use and media practice. In other words, the aim of the proposed methodology is to build on the differences between media use and media practices. As Mathieu (2023) writes: “Usage and practices are two sides of the same coin, but the distinction is nevertheless an important one. … Usage is both the object of measurement and something that can be contained by technology. Practices are neither of these things. Practices are shaped by meaning, tradition, habit, culture and identity, and populate the sociocultural everyday life of audiences.” (p. 35) We argue that user perspectives and digital trace data should not be considered as separate methods but as part of a reciprocal exchange and a broader methodological pluralism (Danermark et al., 2019). Based on use cases of this method, we note that such an approach can demand new skills of researchers, and involve more direct engagement with social media users. However, such an approach is increasingly necessary in the context of highly personalized platforms and the limitations of data access. Moreover, we propose that inviting users into the quantitative analysis process can help correct for the noted lack of agency users have had in big data studies (Bishop & Kant, 2023). Background In the course of our own research, we have iteratively developed a method that combines trace data – from APIs, data donations, or manual collection – with user interviews. This emergent approach has proven to be an effective strategy for on the one hand quantifying media use, while also breathing life into the data by giving voice to the users. Yet, we find little in the methodological literature that provides a clear framework for an alloy between these types of data. While the combination of qualitative interviews with quantitative data traditionally falls under the catch-all “mixed methods,” we find this category provides insufficient methodological backing for social media researchers seeking to contextualize user-level practices. Mixed methods often progress from qualitative to quantitative, with individual interviews informing the structure of subsequent generalizable social surveys, or vice versa, with qualitative interviews providing more social context for the findings of surveys (Creswell & Creswell, 2018). Even when qualitative and quantitative data are collected in tandem,these are not generally data on the same individual. Moreover, such triangulation approaches are often used to corroborate different measures of similar outcome variables (such as user perceptions). These do not take advantage of the availability of digital trace data, the by-products users’ interactions with digital media, offering a window – albeit limited – into user behavior (Rogers, 2019).In the past few years, new methodologies such as data donations have been developed to study social media use, and are often compared epistemologically to self-reports, logs, and phone tracking (van Driel et al., 2022; Ohme et al., 2023). A recent line of literature has examined best practices for obtaining and analyzing donated data (Keusch et al., 2024; Gomez Ortega et al., 2023; Pfiffner & Friemel, 2023), yet we find so far little scholarly writing has addressed combining user data with interviews. We argue that these quantitative trace data alone are insufficient; users are not their data. Our proposed framework suggests collecting digital traces and explanations from the same users, who are brought into the data collection and the analysis. A core premise of our proposed model is that the patterns identified through data scraping and analysis are devoid of meaning without the interpretive voices of the users themselves (Lomborg & Bechmann, 2014). This perspective aligns philosophically in many ways with digital ethnography, but we find that the current scope of digital ethnography does not lend itself to quantitative explorations of digital trace data. Ethnography is generally a qualitative approach that emphasizes the reflexivity of the researcher and the in-depth exploration of cultural milieux (Coleman, 2010). Computational analysis of digital trace data is difficult to fit into the traditions of digital ethnography. Thus, our methodological proposal is not a mere academic exercise; it is a practical advancement of existing methods, and an adaptation to the evolving landscape of digital interaction. We demonstrate the application of the reciprocal model through two use cases: Spotify and Twitter/X. Case study I: Twitter/X In the first case study, the reciprocal approach was applied to the study of communication in global networks on Twitter, involving interviews with 26 users. In advance of each interview, the user’s tweet archive was accessed through the platform's API – acknowledging that profile scraping could serve as an alternative – and the researcher identified the user’s most popular tweets and mapped patterns of communication. During the subsequent interview, tweets were presented to the participant. This technique prompted users to engage in more specific and reflective discussions about their tweeting practices, including the temporal patterns of their activity. This reciprocal method not only allowed users to clarify and correct the researchers’ interpretations, but also revealed insights into phenomena such as the differentiation between disengagement and user fatigue, and the repercussions of platform suspension (visible in the trace data only as periods of reduced tweeting). This reflective process underscored the value of integrating user perspectives into metadata analysis of digital trace data. Case study II: Spotify The second case study focused on Spotify, with an emphasis on the interaction between users and the recommendation algorithm. The researcher first interviewed 20 participants to gather qualitative insights into their perceptions and attitudes towards the platform and its algorithmic recommendations. Following these discussions, participants were asked to provide “donated data” – logs of their Spotify usage. This combination of qualitative narratives with quantitative usage data allowed the researcher to discern discrepancies between how users perceived their interactions with Spotify's algorithm and their actual engagement patterns as evidenced in the donated data. It revealed not only patterns for music listening and discovery but also tensions between self- perceptions and listening behavior. Through this reciprocal process, the study revealed a complex relationship between user agency and algorithmic influence. Toward reciprocal digital methods Some fields, projects, and research questions will naturally benefit from purely quantitative or qualitative approaches. Our aim is to provide a framework for inquiries that seek to bridge user behavior (use) and user perception (practice). Reciprocal digital methods addresses the importance of, on the one hand, not looking at digital data in isolation, and on the other, grounding interviews in specific use patterns – ultimately providing a more complete picture of the digital experience. References Bishop, S., & Kant, T. (2023). Algorithmic autobiographies and fictions: A digital method. The Sociological Review, 71(5), 1012- 1036. https://doi.org/10.1177/00380261221146403 Bruns, A. (2019). After the ‘APIcalypse’: Social media platforms and their fight against critical scholarly research. Information, Communication & Society, 22(11), 1544-1566. https://doi.org/10.1080/1369118X.2019.1637447 Coleman, E. G. (2010). Ethnographic Approaches to Digital Media. Annual Review of Anthropology, 39, 487–505. http://www.jstor.org/stable/25735124 Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative & mixed methods approaches. (5th ed.). Sage. Danermark, B., Ekström, M., & Karlsson, J. Ch. (2019). Explaining society: Critical realism in the social sciences. Routledge. Freelon, D. (2018, 2018/10/02). Computational research in the post-API age. Political Communication, 35(4), 665-668. https://doi.org/10.1080/10584609.2018.1477506 Gomez Ortega, A., Bourgeois, J., Hutiri, W.T. & Kourtuem, G. (2023) Beyond data transactions: a framework for meaningfully informed data donation. AI & Society. https://doi.org/10.1007/s00146-023-01755-5 Keusch, F., Pankowska, P. K., Cernat, A., & Bach, R. L. (2024). Do You Have Two Minutes to Talk about Your Data? Willingness to Participate and Nonparticipation Bias in Facebook Data Donation. Field Methods. https://doi.org/10.1177/1525822X231225907 Lomborg, S., & Bechmann, A. (2014). Using apis for data collection on social media. The Information Society, 30(4), 256-265. https://doi.org/10.1080/01972243.2014.915276 Marres, N., & Gerlitz, C. (2016). Interface methods: Renegotiating relations between digital social research, STS and sociology. The Sociological Review, 64(1), 21–46. https://doi.org/10.1111/1467-954X.12314 Mathieu, D. (2023). Deconstructing the Notion of Algorithmic Control over Datapublics. In DataPublics (pp. 27–48). Bristol University Press. https://bristoluniversitypressdigital.com/display/book/9781529228649/ch002.xml https://doi.org/10.1177/00380261221146403 https://doi.org/10.1080/1369118X.2019.1637447 http://www.jstor.org/stable/25735124 https://doi.org/10.1080/10584609.2018.1477506 https://doi.org/10.1007/s00146-023-01755-5 https://doi.org/10.1177/1525822X231225907 https://doi.org/10.1080/01972243.2014.915276 https://doi.org/10.1111/1467-954X.12314 https://bristoluniversitypressdigital.com/display/book/9781529228649/ch002.xml Ohme, J., & Araujo, T. (2022). Digital data donations: A quest for best practices. Patterns, 3(4). https://doi.org/10.1016/j.patter.2022.100467 Ohme, J., Araujo, T., Boeschoten, L., Freelon, D., Ram, N., Reeves, B. B., & Robinson, T. N. (2023). Digital Trace Data Collection for Social Media Effects Research: APIs, Data Donation, and (Screen) Tracking. Communication Methods and Measures, 0(0), 1–18. https://doi.org/10.1080/19312458.2023.2181319 Pfiffner, N., & Thomas. N. Friemel (2023) Leveraging Data Donations for Communication Research: Exploring Drivers Behind the Willingness to Donate. Communication Methods and Measures, 17(3), 227- 249. https://doi.org/10.1080/19312458.2023.2176474 Rogers, R. (2019). Doing digital methods. Sage. Tromble, R. (2021). Where Have All the Data Gone? A Critical Reflection on Academic Digital Research in the Post-API Age. Social Media + Society, 7(1). https://doi.org/10.1177/2056305121988929 van Driel, I. I., Giachanou, A., Pouwels, J. L., Boeschoten, L., Beyens, I., & Valkenburg, P. M. (2022). Promises and Pitfalls of Social Media Data Donations. Communication Methods and Measures, 16(4), 266–282. https://doi.org/10.1080/19312458.2022.2109608 https://doi.org/10.1016/j.patter.2022.100467 https://doi.org/10.1080/19312458.2023.2181319 https://doi.org/10.1080/19312458.2023.2176474 https://doi.org/10.1177/2056305121988929 https://doi.org/10.1080/19312458.2022.2109608