Selected Papers of #AoIR2024: The 25th Annual Conference of the Association of Internet Researchers Sheffield, UK / 30 Oct - 2 Nov 2024 Suggested Citation (APA): Mitchell, P., Riedlinger, M., Burgess, J., Snoswell, A., Wizenberger, K., & Goldenfein, J. (2024, October). Research GenAI: Situating generative AI in the scholarly economy. Paper presented at AoIR2024: The 25th Annual Conference of the Association of Internet Researchers. Sheffield, UK: AoIR. Retrieved from http://spir.aoir.org. RESEARCH GENAI: SITUATING GENERATIVE AI IN THE SCHOLARLY ECONOMY Peta Mitchell Queensland University of Technology Michelle Riedlinger Queensland University of Technology Jean Burgess Queensland University of Technology Aaron Snoswell Queensland University of Technology Kevin Witzenberger Queensland University of Technology Jake Goldenfein University of Melbourne Introduction This paper focuses on a small but growing subset of GenAI tools and platforms designed for and explicitly marketed to academic researchers, which we call Research Generative AI or RGAI. ChatGPT’s release in late 2022 initiated significant and ongoing debate over GenAI’s implications for academic research and publishing. This debate, along with the academic publishing and institutional research policies established in its wake, has largely focused on generic GenAI platforms (such as ChatGPT and Midjourney). Less attention has been paid to emergent RGAI platforms, such as Consensus, Elicit, Perplexity, Iris.ai, Scholarcy, Scite, SciSpace, and Writefull, which unambiguously announce their academic focus (see Fig. 1) and promise to automate research discovery and writing tasks, such as identifying and summarising published research, writing literature reviews, conducting data analysis, and synthesising findings. https://consensus.app/ https://elicit.com/ https://www.perplexity.ai/ https://iris.ai/ https://www.scholarcy.com/ https://scite.ai/ https://scispace.com/ https://www.writefull.com/ Many of these RGAI platforms pre-date the public release of ChatGPT, but their development and uptake has been propelled by the current GenAI moment, and many are built on or have been redeveloped to incorporate GPT foundational models. Figure 1: Homepages of Research GenAI platforms In this paper, we present a case for understanding these platforms as a distinct category of GenAI. We do this by 1) mapping the history and development of RGAI platforms and developing a preliminary typology of RGAI, 2) situating RGAI tools and platforms within the scholarly economy and ongoing processes of platformisation and automation of academic work, and 3) making a case for the need to understand RGAI platforms as complex sociotechnical systems that intersect with social, ethical, institutional, and legal questions. RGAI as communicative AI within the scholarly economy We consider RGAI in the context of what Hepp et al. (2023) refer to as “communicative AI”: a “sensitising concept” that assists in drawing focus to the implications for societal communication posed by AI’s automation of communication. Hepp et al.’s (2023) definition of communicative AI is framed around three criteria, namely that a communicative AI system is understood to be 1) “based on various forms of automation designed for the central purpose of communication,” 2), “embedded within digital infrastructures,” and 3) “entangled with human practices” (p. 47). Through our analysis of RGAI as communicative AI, we bring to this definition—with its focus on automation, digital infrastructures, and human practices—insights into the role of institutions and institutional economies on the development and implications of communicative AI. To this end, we situate RGAI within what has been called the “scholarly economy” (e.g., Goldenfein & Griffin, 2022; Hyland, 2023), joining other forms of “platform capitalism” reshaping universities and academic research (Mirowski 2018; Mirowski 2023). Previous research has examined how digital platforms such as Google Scholar, academia.edu, and ResearchGate increasingly mediate academic research discovery, dissemination, and citation practices (e.g., Goldenfein & Griffin, 2022; Darvin, 2022), thereby contributing to and shaping the scholarly economy. Through their freemium pricing models, their endorsement and promotion by academic AI “thought leaders,” and their growing inclusion on lists of recommended research tools developed by university libraries, RGAI platforms are becoming embedded within the scholarly economy and, we argue, contributing to the growth and complexity of automation and platformisation of scholarly research and publishing. Within the broader debate over GenAI’s potential impact on universities and academic research and publishing, there has been limited research that places the debate in the context of automation. Although published research on GenAI, authorship, and academic publishing acknowledges the heightened risks of algorithmic bias, lack of transparency and accountability, and reduced discoverability that attend GenAI technologies (e.g., Bell, 2023; Oduoye et al., 2023) the debate is rarely situated in relation to broader research on the ongoing automation and platformisation of academic research and publishing. Watermeyer et al.’s (2023) article ‘Generative AI and the automating of academia’ is a rare instance of work that directly speaks to GenAI’s role in the ongoing automation of academic labour. The study’s findings, based on a survey of UK academics and their use of Large Language Models (LLMs) “like ChatGPT,” reveal that academics tend to employ LLMs as a “labour accelerator” with the overarching aim of “alleviat[ing] their precarity” within a highly competitive academic system (Watermeyer et al. 2023). While Watermeyer et al. (2023) draw important attention to GenAI’s potential influence on the automation of academic labour and the scholarly economy, they do not engage with the specific implications for academic research and publishing raised by the emergent political economy of RGAIs, which this paper addresses. Additionally, no research to date has engaged with RGAI platforms as complex sociotechnical systems. To fill this gap, we employ and adapt Light, Burgess & Duguay’s (2016) walkthrough methodology to compare two Research GenAI (RGAI) tools—Consensus and Writefull—focusing on their roles in academic research, functionality and governance. Comparative walkthrough of Consensus and Writefull Both tools were in existence before the public release of ChatGPT, with Consensus being launched in 2019 and Writefull in 2014. Consensus is categorised as a research discovery tool, designed to streamline literature searches for academia and medical professionals. It uses a proprietary LLM model combined with OpenAI's technology to provide insights into scientific consensus via a “Consensus Meter,” classifying search results into categories such as “yes,” “no,” and “possibly” based on sentiment analysis. This tool appeals to efficiency-focused researchers by promising rapid, evidence-based insights while emphasising responsible AI use. In contrast, Writefull, which caters to academic writing support, aims to simplify what is described as the often-challenging task of scholarly writing. It offers features like writing, paraphrasing, and copyediting— positioning the tool as a task-specific solution for early-career researchers and non- native English speakers. Writefull employs custom AI models tailored to academic language, incorporating tools that are optimised for tasks like generating abstracts. Its “Academizer” tool formalises language through nominalisations and passive constructions, reinforcing traditional normative standards of academic writing. Writefull markets itself as assisting writers in meeting established publication standards, with clients like the American Chemical Society integrating it into their editorial workflows. Both tools present themselves as independent and somewhat bespoke yet they are backed by substantial corporate entities. Consensus, developed by a team with roots in prominent technology firms and funded by venture capital, achieved a valuation of USD$15.7 million in 2023. Writefull, initially founded by linguists and AI researchers, is now owned by Digital Science, a subsidiary of Holtzbrinck Publishing Group, which oversees influential platforms like Altmetric and Dimensions, and academic publishers like Springer Nature and Macmillan. This ownership aligns Writefull with the entire academic publishing process, aiding researchers, journals and publishers in maintaining quality standards for a fee. In future work, we will examine these and other RGAI tools through surveys and ethnographic studies, aiming to understand their cultures of use within academia. The team is also developing an RGAI Index prototype to guide researchers in selecting suitable AI tools based on comprehensive assessments of their features, functionality, and governance. References Bell, S. (2023). The write algorithm: Promoting responsible artificial intelligence usage and accountability in academic writing. BMC Medicine, 21(1), 334. https://doi.org/10.1186/s12916-023-03039-7 Committee on Publication Ethics. (2023, February 13). COPE position statement on Authorship and AI tools. https://publicationethics.org/cope-position-statements/ai- author Darvin, R. (2021). The digitally-mediated scholar: Online identities and investment in the discursive practices of an academic community. In P. Habibie & S. Burgess (Eds.), Scholarly publication trajectories of early-career scholars. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-030-85784-4_10 Goldenfein, J. & Griffin, D. (2022). Google Scholar – Platforming the scholarly economy. Internet Policy Review, 11(3). https://doi.org/10.14763/2022.3.1671 Hepp, A., Loosen, W., Dreyer, S., Jarke, J., Kannengießer, S., Katzenbach, C., Malaka, R., Pfadenhauer, M., Puschmann, C., & Schulz, W. (2023). ChatGPT, LaMDA, and the hype around communicative AI: The automation of communication as a field of research in media and communication studies. Human–Machine Communication, 6. https://doi.org/10.30658/hmc.6.4 https://publicationethics.org/cope-position-statements/ai-author https://publicationethics.org/cope-position-statements/ai-author https://doi.org/10.1007/978-3-030-85784-4_10 https://doi.org/10.14763/2022.3.1671 https://doi.org/10.30658/hmc.6.4 Hyland, K. (2023). Academic publishing and the attention economy. Journal of English for Academic Purposes, 64. https://doi.org/10.1016/j.jeap.2023.101253 Light, B., Burgess, J., & Duguay, S. (2016). The walkthrough method: An approach to the study of apps. New Media & Society, 20(3). https://doi.org/10.1177/1461444816675438 Mirowski, P. (2018). The future(s) of open science. Social Studies of Science, 48(2), 171–203. https://doi.org/10.1177/0306312718772086 Mirowski, P. (2023). The evolution of platform science. Social Research, 90(4), 725– 755. https://doi.org/10.1353/sor.2023.a916352 Nature Editorials. Tools such as ChatGPT threaten transparent science; here are our ground rules for their use. Nature, 613. https://doi.org/10.1038/d41586-023-00191- 1 Oduoye, M. O., Javed, B., Gupta, N., & Sih, C. M. V. (2023). Algorithmic bias and research integrity; The role of nonhuman authors in shaping scientific knowledge with respect to artificial intelligence: a perspective. International Journal of Surgery, 109(10), 2987–2990. https://doi.org/10.1097/JS9.0000000000000552 Watermeyer, R., Phipps, L., Lanclos, D., & Knight, C. (2023). Generative AI and the automating of academia. Postdigital Science and Education, 6, 446–466. https://doi.org/10.1007/s42438-023-00440-6 World Association of Medical Editors. (2023, January 20). WAME recommendations on ChatGPT and chatbots in relation to scholarly publications. https://wame.org/page3.php?id=110 https://doi.org/10.1016/j.jeap.2023.101253 https://doi.org/10.1177/1461444816675438 https://doi.org/10.1177/0306312718772086 https://doi.org/10.1353/sor.2023.a916352 https://doi.org/10.1038/d41586-023-00191-1 https://doi.org/10.1038/d41586-023-00191-1 https://doi.org/10.1097/JS9.0000000000000552 https://doi.org/10.1007/s42438-023-00440-6 https://wame.org/page3.php?id=110