Selected Papers of #AoIR2022: The 23rd Annual Conference of the Association of Internet Researchers Dublin, Ireland / 2-5 Nov 2022 Suggested Citation (APA): Chia, A., Malazita, J., Young, C., Nieborg, D., Joseph, D., Gantt, M. (2022, November). “The Engine is the Message: Videogame Infrastructure and the Future of Digital Platforms”. Panel presented at AoIR 2022: The 23rd Annual Conference of the Association of Internet Researchers. Dublin, Ireland: AoIR. Retrieved from http://spir.aoir.org. THE ENGINE IS THE MESSAGE: VIDEOGAME INFRASTRUCTURE AND THE FUTURE OF DIGITAL PLATFORMS Aleena Chia Goldsmiths, University of London James Malazita Rensselaer Polytechnic Institute Chris J. Young University of Toronto Mississauga David B. Nieborg University of Toronto Scarborough Daniel J. Joseph Manchester Metropolitan University Matthew D. Gantt Rensselaer Polytechnic Institute On January 18, Microsoft revealed its $68.7 billion deal to acquire videogame publisher Activision Blizzard. Journalists speculated about the increasing consolidation of the videogame industry that would make the software giant the third-largest game company in the world by revenue, after Tencent and Sony (Sweney 2022). The acquisition was pitched as an investment towards “metaverse platforms” that gaming would play a key role in developing (Microsoft News Center 2022). The metaverse is a concept adapted from cyberpunk fiction and re-envisioned by tech companies as persistent, real-time networked 3D environments for people to connect, socialize, play, and work together (Bell 2022). Pivoting away from the hype drummed up by Facebook’s (now known as Meta) transition from a social media company to a "metaverse company", many commentators focused on the implications of Microsoft’s acquisition for content distribution and work conditions. Reporting focused instead on whether blockbuster franchises like Call of Duty would be locked into Microsoft’s platforms and subscription services (Keogh 2022), and whether changes in Activision Blizzard’s management would improve a toxic workplace 2 culture that ignored sexual harassment and abuse (Skrebels 2022). At the same time, reporting on sexual harassment in Meta’s VR platform Horizon Worlds, tech journalists flagged continuities and consternations over trust and safety in virtual worlds (Hendrix 2022). Seemingly above the fray of platform strategy, market speculation, and corporate scandal, New Yorker writer Kyle Chayka (2022) tweeted as a matter of fact: “video game infrastructure and tools are increasingly going to take over all digital platforms”. This panel contextualizes these buzzworthy developments, future imaginaries, and escalating controversies in the “boring things” (Star 1999, 377) required to build and maintain the metaverse, such as the infrastructure of physics simulation and real-time rendering that have been pioneered by game engines. Early internet studies probed how difference was articulated sociotechnically in “cyberspace” (Kolko, Nakamura, and Rodman 2000). Research on “virtual worlds” investigated how governance of online embodiment and interaction was enacted through design of game mechanics and environments (Boellstorff 2008; Malaby 2009; Nardi 2010). Platform studies critiques how online content is regulated and power is consolidated in platform corporations (Poell, Nieborg, and Duffy 2021). The sociotechnical focus of these intersecting lines of inquiry informs our critical interest in the infrastructure of the metaverse: how are the exigencies of the future metaverse currently being laid in game engines? And how are these futures enmeshed with technolibertarian fantasies of a decentralized, purchasable web, John Perry Barlow’s “Independent Cyberspace” cast through a hedge fund manager’s lens? Our papers ground discussions about the business and aesthetics of 3D platforms in the infrastructural work of game engines, which integrate databases, file formats, web protocols, and translational algorithms. We trace debates over representation and governance, equity and inclusion (Bosworth 2021) to the techniques, technologies, and practices that enable massive real-time 3D digital spaces to flow and transact. We also highlight the growing intertwinement between game engine development companies and related content ecosystems, such as the Epic Games Store and the Unreal Engine, and Epic’s and Unity’s Asset Stores. This panel investigates how digital systems are designed to regulate technical interoperability and its implications for creative practice and cultural production. Together, these papers map how power and capital become centralized and distributed throughout the back end of the metaverse, and politicize how social practices and subjectivities are negotiated through technological architecture. The first paper “Shaping Cultural Production” charts a history of Unity Technologies’ acquisitions and partnerships with companies in the RT3D immersive production pipeline. By documenting Unity’s expansion into adjacent industry sectors and its integration within the infrastructures of dominant platform companies, this paper argues that the engine development company’s platformization of cultural production points to further colonization of platform data and intensification of platform imperialism. The second paper on “Virtual Creative Practice” contextualizes corporate visions of the metaverse in a longer history of grassroots experimentation in 3D worldbuilding. By juxtaposing the brash futurism of platform companies with a survey of creative pioneers working with networked digital platforms, this paper suggests alternative pathways towards immersive media futures. Through tracing Fortnite and Unreal Engine owner Epic Games’ hybrid technical- economic metaverse strategies, the third paper will argue that Epic Games highlights the ontological uncertainties facing platform studies and game studies in the coming decade, 3 as the “layers model” of production—where the possibilities of play are in part determined by the technologies underneath, becomes increasingly untenable. The final paper “The Future will be Captured” explores the interplay between race and the politics of realism in RT3D engine tools that are positioned to populate the metaverse with “digital humans.” By analysing the sociotechnical assumptions of Epic’s 3D human scanning, this paper critiques these tools’ prescriptions of epidermal variation and phenotypical precision and its implications for representational diversity in RT3D character creation. References Bell, Genevieve. 2022. “The metaverse is a new word for an old idea.” MIT Technology Review. 8 February. https://www.technologyreview.com/2022/02/08/1044732/metaverse-history-snow- crash/ Boellstorff, Tom. 2008. Coming of Age in Second Life: An Anthropologist Explores the Virtually Human. New Jersey: Princeton University Press. Bosworth, Andrew. 2021. “Building the Metaverse Responsibly.” Meta. 27 September. https://about.fb.com/news/2021/09/building-the-metaverse-responsibly/ Chayka, Kyle. 2022. “video game infrastructure and tools are increasingly going to take over all digital platforms”. 18 January. Twitter. https://twitter.com/chaykak/status/1483490290719764483 Hendrix, Justin. 2022. “The Sunday Show: Trust and Safety in Virtual Worlds.” Tech Policy Press [Podcast]. 6 February. https://techpolicy.press/the-sunday-show-trust- and-safety-in-virtual-worlds/ Keogh, Brendan. 2022. “Microsoft buys Activision Blizzard: with the video game industry under new management, what’s going to change?” 23 January. The Conversation. https://theconversation.com/microsoft-buys-activision-blizzard-with-the-video- game-industry-under-new-management-whats-going-to-change-175433 Kolko, Beth, Lisa Nakamura, and Gilbert Rodman. 2000. Race in Cyberspace. New York: Routledge. Malaby, Thomas. 2009. Making Virtual Worlds: Linden Lab and Second Life. Ithaca: Cornell University Press. Microsoft News Center. 2022. “Microsoft to acquire Activision Blizzard to bring the joy and community of gaming to everyone, across every device.” January 18. https://news.microsoft.com/2022/01/18/microsoft-to-acquire-activision-blizzard-to- bring-the-joy-and-community-of-gaming-to-everyone-across-every-device/ Nardi, Bonnie. 2010. My Life as a Night Elf Priest: An Anthropological Account of World of Warcraft. Ann Arbor: The University of Michigan Press. 4 NVIDIA. 2021. “Plumbing the Metaverse with USD.” GTC Digital April. Available at: https://www.nvidia.com/en-us/on-demand/session/gtcspring21-s31946/ Accessed 9 November 2021. Poell, Thomas, David B. Nieborg, and Brooke Erin Duffy. 2021. Platforms and Cultural Production. Cambridge, UK: Polity. Skrebels, Joe. 2022. “Update: Bobby Kotick’s future at Activision Blizzard after Xbox acquisition remains unconfirmed.” 24 January. IGN. https://www.ign.com/articles/bobby-kotick-xbox-activision-blizzard-acquisition- update Star, Susan Leigh. 1999. “The Ethnography of Infrastructure.” American Behavioral Scientist, 43(3): 377–391. Sweney, Mark. 2022. “Microsoft takeover of Call of Duty maker wipes $20bn off Sony shares.” 19 January. The Guardian. https://www.theguardian.com/technology/2022/jan/19/microsoft-takeover-call-of- duty-games-firm-wipes-20bn-off-sony-shares-playstation-activision-blizzard-games 5 SHAPING CULTURAL PRODUCTION: UNITY TECHNOLOGIES PAN- INDUSTRY INFRASTRUCTURAL INTEGRATION Chris J. Young University of Toronto Mississauga David B. Nieborg University of Toronto Scarborough Daniel J. Joseph Manchester Metropolitan University Late 2021, Unity Technologies acquired the visual effects (VFX) company Weta Digital. Unity is best known for the Unity software suite that produces real-time 3D (RT3D) immersive content across the video game industry, and more recently the film, automotive, and architecture industries. The acquisition included a slew of proprietary technologies, as well as VFX artists and engineers who have been responsible for producing the tools and development pipelines that brought to life iconic franchises like Lord of the Rings, Game of Thrones, and Wonder Woman. Leveraging its capital position following its November 2020 initial public offering, buying Weta underscores the company’s ambition to leverage industry-specific software to deploy across multiple industries (Young, 2021). As stated in a press release “once integrated onto the Unity platform” Weta’s sophisticated tools “enable the next generation of RT3D creativity and shape the future of the metaverse” (Business Wire 2021). Rather than directly competing with platform companies such as Meta, Google, and Microsoft, which aim to host and monetize the metaverse, Unity’s ambition is to supply the very tools necessary to build immersive virtual spaces. In this paper we retrace the history of Unity from a small Danish game developer to a US- based technology platform. We argue that the company’s trajectory, ambitions, and business operations are emblematic of what has been theorized as the “platformization of cultural production”: Unity may have “radically lowered the bar for teams of all sizes to create virtual worlds,” via its “infrastructural integration” with dominant distribution platforms – e.g., app stores and game consoles – Unity has also made the development of virtual assets “platform-dependent” (Poell et al., 2021: 194-5). This level of institutional dependency has two political economic implications. First, Unity’s growth strategy follows the trajectory of US-based platform companies that have made loss-leading investments to assert or retain market dominance. Second, the current crop of dominant platform companies are collectively implicated in the process of “platform imperialism” (Jin, 2015), wherein a wider system of economic and cultural imperialism is driven by corporate and finance capital to export to markets in the colonial and imperial periphery. For example, Unity’s imperial ambitions become apparent in its close collaboration with the United States military’s “GovTech” division (Klepek, 2021). More broadly, its toolsets are explicitly positioned to be infrastructurally integrated with US-based companies that aim to own and operate the metaverse. As a result, any serious attempt to decolonize the internet, or any future incarnations of ‘the metaverse’ for that matter, must account for these imperial dynamics and the uneven flows of capital and culture associated with them. 6 Methods To provide further insight into how these two dynamics have unfolded over time, we deploy a mixed method approach. First, we gathered a corpus of journalistic accounts and company documents from 2004 to 2022. The news and financial databases of Bloomberg, Factiva, and FactSet were searched to gather press releases and news clips on Unity Technologies. Second, we gathered a corpus of company documentation using the Internet Archive’s Wayback Machine to collect press releases, patch notes and updates to Unity software and services, documentation on integration of third-party software, blogs and announcements from executive officers, and newspaper reporting on Unity acquisitions and use cases. This corpus allows us to reconstruct a timeline of notable acquisitions, business partnerships, cross-platform support, features, and updates that speak to the processes of corporate dominance and infrastructural integration. Periodization Our periodization breaks down into four overlapping stages. First, for the longest time (2004 - 2016) Unity Technologies was primarily positioned to be a production platform to produce games. The introduction of game development pipeline features, such as the Asset Store, Unity Ads, Unity Cloud, and Unity Connect, enabled game creators to develop, market, and monetize one’s games. The acquisition of companies like Applifier and Multiplay in 2014 and 2017 supported this business strategy by integrating essential game technologies, such as multiplayer hosting and instant replays. Second, we witnessed sectoral expansion. In 2014, Unity began to integrate virtual reality (VR) and augmented reality (AR) features, which infrastructurally integrate Unity with companies in industries looking for real-time solutions: e.g., virtual condominium walkthroughs or car demos. The company subsequently engaged in several business partnerships to further expand into segments outside of the entertainment industry by acquiring VisualLive to integrate AR for building information modeling and Pixyz to optimize 3D automotive data. Third, starting in 2017, Unity acquired 21 companies with industry-specific tools, assets, and workflows to transform its game engine into a pan-industry production platform. Most of these acquisitions concern companies that originated in the game industry but have come to provide more generic services, such as cloud-based remote desktop for virtual collaboration (Parsec), product visualization for marketing (RestAR), and version control for proprietary formats and metadata (Codice Software). Alongside the VFX acquisition of Weta Digital, we argue that Unity is close to being unrivaled in its ability to provide RT3D-related services to industries across the globe. Fourth, Unity has long standing strategic business partnerships with Meta, Google, and Microsoft, to position itself as the metaverse production platform of choice. Instead of competing head on with the dominant platform companies, Unity has made every effort to further integrate its production infrastructures with dominant platform companies. Customers of Unity’s partners can utilize their custom scripting tools, modular application 7 architecture, and cloud-based services to build and monetize an entire RT3D immersive experience. What our periodization illustrates is that Unity has leveraged its dominant position in the game industry to rapidly acquire and partner with companies in the RT3D immersive production pipeline to expand into adjacent industry sectors and to solidify its integration within the infrastructures of dominant platform companies. Our analysis contributes to the field of platform studies, specifically the growing area of scholarship that focuses on production platforms. These studies have predominantly analyzed Unity and point to its role in game development (Chia et al, 2020; Nicoll & Keogh, 2019; Young, 2021) and how Unity functions as a multi-sided (or platform) market (Foxman, 2019). Our contribution is meant to broaden the conversation about Unity’s unique role as a production platform in the platformization of cultural production. Unity Technologies’ corporate expansion and infrastructural integration across industries point to further colonization of platform data and the intensification of platform imperialism. References Business Wire. 2021. “Unity Announces Intent to Acquire Weta Digital.” Business Wire. 9 November. https://www.businesswire.com/news/home/20211109006514/en/ Chia, A., Keogh, B., Leorke, D., & B. Nicoll. (2020). Platformisaton in Game Development. Internet Policy Review, 9(4): 1-28. doi:10.14763/2020.4.1515 Foxman, Maxwell. (2019). United We Stand: Platforms, Tools and Innovation with the Unity Game Engine. Social Media + Society, 5(4), 1–10. doi:10.1177/2056305119880177 Jin, D. Y. (2015). Digital Platforms, Imperialism and Political Culture. Routledge. Klepek, P. (2021, August 23). Unity Workers Question Company Ethics As It Expands From Video Games to War. Vice. https://www.vice.com/en/article/y3d4jy/unity- workers-question-company-ethics-as-it-expands-from-video-games-to-war Nicoll, B. & B. Keogh. (2019). The Unity Game Engine and the Circuits of Cultural Software. Palgrave Macmillan. Poell, T., Nieborg, D. B., & Duffy, B. E. (2021). Platforms and Cultural Production. Polity. Young, C. J. (2021). Unity Production: Capturing the Everyday Game Maker Market. In O. Sotamaa & J. Švelch (Eds.). Game Production Studies (pp. 141–58). Amsterdam: University of Amsterdam Press. 8 VIRTUAL CREATIVE PRACTICE: IMMERSIVE WORLDS, PLATFORM ART AND AFFORDANCE, METAVERSE FUTURES Matthew D. Gantt Rensselaer Polytechnic Institute While recent discourse surrounding ‘the metaverse’ is often connected to Neal Stephenson’s 1992 novel Snow Crash, notions of expansive, experimental, or immersive environments have long been a point of cultural fascination, even before the term itself was coined (Bell 2022). Similarly, artistic experiments with digitally navigable space, networked virtual environments, and ‘telematic’ performance offer a rich history and strong precedent for critical inquiry into the possibilities of immersive or networked media space(s). This paper will identify creative pioneers working with networked digital platforms, investigate related approaches in contemporary artistic practice in immersive worlds, and point towards future developments. This investigation will focus not just on a teleology of artistic works or practitioners, but rather, an exploration of the relationship between practitioners and the platforms they become entangled with. The Art Happens ‘Here’ The canonical Simple Net Art Schematic by the artist duo MTAA contains an illustration of two computers connected by a line, stating that the art happens ‘here’, or ‘in’ the space of connection. This notion of networked art as a dynamic ‘thing’ rather than an immutable artifact remains instrumental in contemporary digital practice (Rhizome 2019). However, the affordances, use cases and embedded ideologies of these networked infrastructures are not universal. As such, creative practice offers a means of going beyond making ‘beautiful’ artworks, and instead becomes a critical investigation into the encoded assumptions of the tools and technologies at play by rupturing, rerouting or reframing their affordances and situated context(s) (Lovejoy et al. 2011). An example of this approach might be seen in the pioneering 1980 work Hole In Space by Kit Galloway and Sherrie Rabinowitz. Repurposing broadcast technology to create a realtime audiovisual stream between storefronts in NYC and LA, the artists offered an early premonition of videochat while simultaneously suggesting a rupture of embodied ‘site’ or space. This repurposing of media as a kind of ‘place’ offers a foreshadowing of our contemporary condition, and challenged previous expectations of ‘where’ we exist in relation to electronic media (Malloy 2016). In the decade following, this notion of networked media as a ‘place’, consumer internet access, early game engines and 3D graphics converged in a proliferation of online ‘worlds’, such as 1995’s WorldsChat or 1996’s Cybertown/Colony City, both built with VRML, a ‘virtual’ answer to HTML. The company Blaxxun Interactive, another digital ‘worldbuilder’ of the era was named after a fictional ‘metaverse nightclub’ in Stephenson’s novel Snow Crash, offering an explicit link to the early zeitgeist imagining these digital platforms as cultural ‘places’ (Murphy and Zehmil 2021). 9 Experimental Practice and ‘Mainstream’ Worldbuilding The following decade brought perhaps the most well known ‘virtual world’, 2003’s Second Life. While extensively documented as a space for online socializing, subcultures and new digital economies (Malaby 2009), Second Life also served as a platform for media artists to negotiate the line between earlier practice and the unique affordances of new digital platforms. An example may be seen in pioneering composer Pauline Oliveros’s work with the ‘virtual ensemble’ Avatar Orchestra Metaverse (AOM) (NWN 2007). While Oliveros had long been invested in the notion of sound in digital space (Oliveros 1995), the ability to inhibit that virtual space makes the connection literal. However, many projects undertaken by Oliveros required a repurposing of the affordances of the platform: despite the open-ended ‘sandbox’ of Second Life, much of the functionality needed to compose or perform ‘virtual music’ was still outside the implicit use case of the platform. As a result, Oliveros and collaborators designed idiosyncratic methods for ‘rerouting’ the affordances of the platform to their ends by writing scores that reframed the platform’s features, or by building unique functionality by reconceptualizing the platform’s interface. Another approach may be seen in Cao Fei’s RMB City, operated in Second Life between 2008 and 2011. Whereas Oliveros stretched the affordances of the platform to entirely new ends, RMB City instead foregrounded the implicit use case of Second Life ad absurdum, creating and surveilling a speculative virtual urban space. In this way, Fei juxtaposes the idealized ‘world building’ of the Second Life platform with the real life ideology of world building in her rapidly urbanizing home in China (Rhizome 2019). These approaches of working ‘with’ or ‘against’ the encoded logics of a platform offer a connection to historic strategies for critical engagement with a medium, and set the groundwork for a new generation of practitioners. Virtual DIY vs Corporate ‘Metaverses’, WebVR The recent confluence of consumer VR, pandemic-driven remote cultural experience, and backlash against contemporary social media has led to an inflection point. While VR has slowly mainstreamed over the last decade, these developments have brought it front and center, often paired with a newfound corporate interest in a loosely defined ‘metaverse’. While many are quick to note the irony that the dystopian ‘metaverse’ of Stephenson’s novel mirrors the problematics of Facebook attempting to strongarm a corporate metaverse into existence, these conversations often overlook the vital experimental work being done with the tools that will comprise the infrastructure of these future worlds. Just as earlier practitioners worked with or against the affordances of their tools, emerging open source WebVR tools such as Mozilla Hubs or A-Frame offer a space for cultural production and critical inquiry. Over the last two years, the Mozilla Hubs platform has proven to be a vital space for this type of work. In contrast to more ‘fully featured’ 10 production tools like the UE4 game engine used to create ‘proto-metaverses’ such as Fortnite, Hubs offers a bare minimum of design affordances, allowing users to build multi- user spaces with embedded media, 3D objects and similar, and little interaction beyond avatar movement or voice chat. However, these limitations have proven generative, as artists have repurposed these simple tools to create virtual galleries, ‘site-specific’ installations and performances, or creative detournements of these platforms to explore new metaphors of what these spaces could be, rather than slavish recreations of high-budget spectacle or existing models (Bicknell-Knight 2021). Similarly, a new generation of artists have begun working with open source tools to build ‘virtual DIY’ spaces and platforms, connecting this kind of exploration to a more utopian ‘bottom up’ type of worldbuilding rather than the ‘top down’ corporate visions of ‘the metaverse’ (Gray Area 2021). Taken together, these creative approaches serve to not only critique existing or emerging technological or cultural developments, but to actuate entirely new visions for immersive media futures. References Bell, Genevieve. 2022. “The metaverse is a new word for an old idea.” MIT Technology Review. 8 February. https://www.technologyreview.com/2022/02/08/1044732/metaverse-history-snow- crash/ Bicknell-Knight, Bob. 2021. Networked Futures: Online Exhibitions and Digital Hierarchies. Isthisit? Gray Area Festival. 2021. “Worlding Protocol.” https://grayareafestival.io/ Lovejoy, Paul, Vesna. 2011. Context Providers: Conditions of Meaning in Media Arts. University of Chicago Press. 1-29. Murphy, Derek and Zemil, Mitchel. 2021 “Preserving Worlds.” Means TV. https://means.tv/programs/preservingworlds?cid=1699447&permalink=mitchell- zemil-pw_pilot_worldschat_h264_210112mp4-15e0e5 Malaby, Thomas. 2009. Making Virtual Worlds: Linden Lab and Second Life. Ithaca: Cornell University Press Malloy, Judy. 2016. Social Media Archeology and Poetics. MIT Press. 166-167. New World Notes. 2007 “Avatar Avant-Garde: Metaverse Orchestra Turns Avatars into Musical Instruments.” https://nwn.blogs.com/nwn/2007/11/avatar-avant-ga.html Oliveros, Pauline. 1995. Acoustic and virtual space as a dynamic element of music. Leonardo Music Journal, 5(1), 19-22. Rhizome. 2019. “RMB CITY.” Net Art Anthology. https://anthology.rhizome.org/rmb-city 11 Rhizome. 2019 “Simple Net Art Diagram.” Net Art Anthology. https://anthology.rhizome.org/simple-net-art-diagram 12 “ASK THAT QUESTION AGAIN IN 12 MONTHS:” FORTNITE AND THE INSTABILITY OF PLATFORM AND CONTENT James Malazita Rensselaer Polytechnic Institute Epic Games, owners of both the popular game Fortnite and the games industry standard development environment Unreal Engine, have long sought to use their hybrid content- and-platform identity to develop a model for the metaverse that centers both multi-user social space as well as creative making and play. Both of these softwares are key to Epic’s strategy, with Fortnite serving as the player-facing community space that will attract metaverse participants, and Unreal operating as both the underlying infrastructure of Fortnite, as well as platformed into Fortnite itself, allowing for the game to make use of the tools of its own creation. This ongoing platformization of Unreal has resulted in a figure-ground swap: creative and design elements of the Unreal Engine are now being repurposed into gameplay and player-led content creations assets in Fortnite. Dispersing Unreal’s creation and monetization affordances into games has technical, legal, and political ramifications which have metastasized in the form of multiple ongoing lawsuits between Epic Games, Google, and Apple over the rights of media platform distributors to monopolize access to software and hardware platforms. Unreal thus serves as a platform for legal battles and games creation, while being simultaneously transformed into content to be served through games-as-platforms—games which were themselves made using Unreal. This talk will argue that Epic Games highlights the ontological uncertainties facing platform studies and game studies in the coming decade, as the “layers model” of production (Bogost and Montfort 2009)—where the possibilities of play are in part determined by the technologies underneath, becomes increasingly untenable (Malazita 2018). Future Promises Over the past three years, a tangential Twitter conversational thread among Fortnite fans, angel investors, and Epic Games CEO Tim Sweeney has inadvertently documented both the aims and failed promises of the platformization of persistent realtime multiuser 3D environments marketed as “the Metaverse.” In 2019, as part of a “Ask a famous person one question” Twitter thread, securities trader and Roundhill CEO Will Hershey tweeted at Sweeney, asking whether or not Sweeney believes that Fortnite, Epic’s popular free- to-play online game, was more of a game or a platform (Hershey 2019). The question was a prescient one; in the prior year, Epic had embarked on a hybrid technical-marketing endeavor to attempt to retain the game’s at the time 250 million registered players. Epic knew that player interest in the game would eventually wane, and that one of the game’s most important demographics—a Twitch-streaming Generation Z—would eventually age out of Fortnite’s fast-twitch, meme-culture inundated gameplay. While Epic would continue—and still continues—to release updates to the game, including new levels, challenges, and player avatar skins, Epic was also interested in turning Fortnite into a more general purpose online hangout and event space, where players of all ages could socialize, as well as stream and create content. Some of this work would entail expanding 13 Fortnite’s “Creative Mode,” an in-game level builder and editor. More would involve cross- corporate partnerships for content creation, which would lead to events like rapper Travis Scott’s in-game March 2020 “Astronomical” concert. In anticipation of Fortnite’s new direction, Sweeney answered Hershey: “Fortnite is a game. But please ask that question again in 12 months.” (Sweeney 2019). Twitter never forgets. A year later, user “Bert” again asked Sweeney about Fortnite’s game/platform status, with Sweeney confirming that “Fortnite is a game… primarily,” but that new updates and functionality in the coming year would push the shooter much closer to what a metaverse platform should be. Not to be deterred, venture capitalist and metaverse hypeman Matthew Ball would ask the same question again at the end of 2021. “Wish I could say platform,” Sweeney replied, “but we need to do a lot more with the tools and economy to actually be there” (Sweeney 2021). Engineering Content This Fortnite mini-saga highlights how the metaverse operates as what Kline, Dyer- Witheford, and de Peuter have called a mutually constitutive circuit of technology, marketing, and culture (2003). Mimicking similar patterns in the cryptocurrency space (Golumbia 2016), capital flows into the metaverse, backed by a series of “just around the corner” promises of full platformization of new or existing 3D environments, only to be stopped in part by the tremendous technical lift involved in developing interoperable software infrastructure. As the games industry has demonstrated, however, solving these infrastructural bottlenecks is not framed only as a matter of technical research and development. Rather, games companies are attempting to buy their way to the metaverse through the acquisition of other media and software companies, betting that the integration of various software tools will create vast leaps forward. Epic Games itself has, over the past several years, purchased volumetric scanning companies, environmental asset databases, motion tracking and animation companies, and--most recently—indie music streaming service BandCamp. Inversely, Epic has also been at the forefront of antitrust legal battles against tech giants like Apple and Google, arguing that platform providers are negatively impacting consumers when they lock down purchasing and installation methods on their hardware. Epic, then, are working both sides of the corporate accumulation and decentralization battle. Additionally, one of Epic’s advantages in the Metaverse space is the Unreal Engine, Epic’s proprietary game engine that is both licensed out to other games companies and also used in house to develop Epic’s own properties—games like Fortnite. Not only is the Unreal Engine an industry standard in games and animation content development, but it is also deeply tied into the Epic Game Store, Epic’s player-facing storefront that both sells games made in Unreal and also sells assets, plugins, and extensions to Unreal itself. In this sense, Epic is already ahead of metaverse competitors like Meta (Facebook), whose business model has long been reliant on users and other developers for content to be deployed on their platforms. Unreal’s infrastructural blurring of online storefront/warehouse also provides a technical advantage to Epic’s metaverse ambitions, as well as a legal and monetary incentive to “open up” other platforms’ payment methods. 14 Epic has been attempting to incorporate the Unreal Engine itself into Fortnite creative mode. Creative mode currently operates akin to games like Roblox, where preprogrammed assets can be recombined in unique and creative ways to make new games and environments, but where designers are ultimately constrained by the limited functionality and availability of the assets themselves. Incorporating the Unreal Engine into Fortnite Creative would allow Fortnite players to build and program their own assets, not only rapidly extending the kinds of play and environments currently possible in the game, and also to sell those assets on the Epic Games store. Game development platforms, then, become reconfigured away from specialist technologies or democratizing systems (Nicoll and Keogh 2019), and instead become game content themselves, inverting games studies’ traditional analytic models of medium-content or layered platforms. References Bogost, Ian, and Nick Montfort. 2009. "Platform studies: Frequently questioned answers." Digital Arts and Culture Golumbia, David. 2016. The Politics of Bitcoin: Software as Right-wing Extremism. University of Minnesota Press Hershey, Will. Twitter Post. December 26, 2019, 3:00 PM. https://twitter.com/maybebullish/status/1210289415991578625?s=20&t=9NnqLR n4_wSOXwG06IRUsQ Kline, Stephen, Nick Dyer-Witheford, and Greig De Peuter. 2003. Digital play: The interaction of technology, culture, and marketing. McGill-Queen's Press-MQUP Malazita, James. "The material undermining of magical feminism in Bioshock infinite: burial at sea." In Feminism in play, pp. 37-50. Palgrave Macmillan, Cham, 2018. Nicoll, Benjamin, and Brendan Keogh. 2019. The Unity Game Engine and the Circuits of Cultural Software. Palgrave Pivot, Cham, 2019. Sweeney, Tim. Twitter Post. December 26, 2019, 3:11 PM. https://twitter.com/TimSweeneyEpic/status/1210291961904730114?s=20&t=9Nn qLRn4_wSOXwG06IRUsQ Sweeney, Tim. Twitter Post. December 26, 2020, 12:04 AM. https://twitter.com/TimSweeneyEpic/status/1342697699179700224?s=20&t=9Nn qLRn4_wSOXwG06IRUsQ Sweeney, Tim. Twitter Post. December 27, 2021, 4:40 PM. https://twitter.com/TimSweeneyEpic/status/1475582484356153349?s=20&t=9Nn qLRn4_wSOXwG06IRUsQ https://twitter.com/maybebullish/status/1210289415991578625?s=20&t=9NnqLRn4_wSOXwG06IRUsQ https://twitter.com/maybebullish/status/1210289415991578625?s=20&t=9NnqLRn4_wSOXwG06IRUsQ https://twitter.com/TimSweeneyEpic/status/1210291961904730114?s=20&t=9NnqLRn4_wSOXwG06IRUsQ https://twitter.com/TimSweeneyEpic/status/1210291961904730114?s=20&t=9NnqLRn4_wSOXwG06IRUsQ https://twitter.com/TimSweeneyEpic/status/1342697699179700224?s=20&t=9NnqLRn4_wSOXwG06IRUsQ https://twitter.com/TimSweeneyEpic/status/1342697699179700224?s=20&t=9NnqLRn4_wSOXwG06IRUsQ https://twitter.com/TimSweeneyEpic/status/1475582484356153349?s=20&t=9NnqLRn4_wSOXwG06IRUsQ https://twitter.com/TimSweeneyEpic/status/1475582484356153349?s=20&t=9NnqLRn4_wSOXwG06IRUsQ 15 THE FUTURE WILL BE CAPTURED: REALITY CAPTURE AND RACIAL CAPITALISM IN REAL-TIME 3D Aleena Chia Goldsmiths, University of London Reality is being captured to create 3D worlds. Photogrammetry scans and matches up points of interest across multiple images of the same physical object to create points in 3D space, which are composited to form a 3D mesh that can be computationally manipulated. Compared to models assembled from wireframes and polygon meshes, interactive and entertainment media industries have increasingly turned to capturing environments through photogrammetry, capturing movement through volumetric techniques, and capturing bodies through 3D human scanning. These techniques automate aspects of modelling, texturing, and animation rigging that require significant human labour. Epic Games has recently acquired photogrammetry studios Quixel and Capturing Reality and digital human startup 3Lateral to build libraries and tools into the Unreal Engine for reconstructing real-time 3D (RT3D) worlds at scale. As the first paper in the panel outlines, engine companies are leading the “platformization of cultural production” to build immersive virtual spaces – and bodies. Today’s discrete worlds built by engines for videogames, visual effects (VFX), architecture, engineering, and construction (AEC) are laying the infrastructure for Meta, Microsoft, and other platform companies’ vision of an RT3D internet “that support[s] continuity of identity, objects, history, payments, and entitlements” (Ball 2021). Engines are gearing up for “pan-industry infrastructural integration” (Panelist 1) to construct and populate the future metaverse. This paper grounds the hype about future worlds in the “plumbing of the metaverse” (Bradshaw and Murphy 2021): 3D modelling, physics simulation, and real-time rendering of game engines. Positioning game engines as infrastructure for 3D rendered assets “enable companies to adopt platform strategies and gain a broader foothold in the digital economy” (Werning 2021: 13). Drawing on textual analysis of promotional and technical documentation and app walkthrough of the Unreal Engine’s reality capture techniques, this paper analyses the interplay between race and realism from scan to rig in digital human tools. The global distribution of the reality capture pipeline – Eastern European toolmakers and Chinese, Indian, and South American asset artists – is laying the infrastructure of the metaverse according to racial capitalism’s (Lowe 2015) differential valuations of labour, resources, and markets. Engines mediate RT3D’s labour and aesthetics as platform and software tools. Digital human software such as Epic’s MetaHuman Creator offers extensive epidermal variation as proof of representational diversity in RT3D character creation. Yet this quantization of race as a proxy for graphic realism must be situated in the infrastructural ambitions of game engines. Automation and Articulation Engines automate aspects of art production in 3D worldbuilding. At the 2017 Game Developers Conference, Naughty Dog’s technical art director Andrew Maximov assured artists that their jobs would not be automated. He advised artists to focus on creatively stylized forms of worldbuilding and leave photorealistic recreations of 3D environments 16 for automation by photogrammetry and engine-based physics simulations. However, the reality capture pipeline is stratified according to racial capitalism and its colonial hierarchies of value: before art assets can be creatively composed into a world, they must be meticulously scanned – a task routinely outsourced to China. Furthermore, the replication of reality in RT3D still requires humans to do the “articulation work” (Star and Strauss 1999) of getting things on track in the face of routine contingencies that are made invisible in automated workflows (Gray and Suri 2019). Automation of 3D art production requires human articulation to tune and tweak outputs from algorithmic processes such as photogrammetry or procedural generation. Yet, articulation work is devalued as mechanical and deemed interchangeable with outsourced work of East and South Asians, who are routinely pegged as less creative and more robotic (Chia 2022). As engines platformize cultural production, the metaverse will be built on sociotechnical infrastructures informed by racial capitalism. The Racialization of Realism Platform companies such as Meta foresee a metaverse embodied through personalized RT3D avatars. Epic provides a scalable solution to digital embodiment: MetaHuman Creator allows designers across videogames, VFX, and AEC to create, manipulate, animate, and light 3D models called digital humans. The free cloud-based app boasts “near-infinite variations of facial features and skin complexions” (Epic Games 2022). At launch, the featured sample of digital humans prominently featured black and East Asian presets. Like other digital human tools, MetaHuman Creator uses 3D scans of real people to populate their database with quantized physical features and movements categorized through racial (and other) presets. Phillips (2020) explains that videogame character customization systems quantize human bodies in ways informed by the racial science of physiognomy. This colonial genealogy of digital humans is complicated by the reality capture infrastructure of engine tools and its ambition towards prescriptive forms of graphical realism. The toolmakers state that “[t]he scan data is the closest thing you’ll get to a ground truth for the subject in 3D” (Unreal Engine 2021). In this context, the slogan of “high-fidelity digital humans” goes beyond the criterion of high polygon count to connote phenotypical precision. Constraints are locked to “ensure accuracy” in creating “physically plausible MetaHumans.” These constraints are not just anatomical but racial. For example, MetaHuman Creator constrains the range of skin tones and hair colours as well as user adjustments “to fit within the limits of the examples in its database” (Epic Games 2022). Many videogames are designed with realism in mind, for example, for players to rate human characters with relevant physical and cognitive abilities. Srauy and Cheney- Lippold (2019) contextualize that even though such realism is framed as neutral code that afford precision and authenticity, this design strategy perpetuates anti-black and anti- Latino racism. Tracing the history of computational images, Gaboury (2021: 9) states that they “are not pictures of the things they represent; they are pictures of the world that produced them, and they execute a theory of that world in the world.” The realism prescribed by RT3D engines execute a theory of the world – its materials, movements, and bodies – that derive from videogame infrastructure and will develop into metaverse platforms. As we anticipate 17 the metaverse, we must politicize its infrastructure that is currently being laid and the alternative configurations it can still take. References Chia, A. 2022. “The artist and the automaton in digital game production.” Convergence, 28(2): 389–412. Ball, Matthew. 2021. “Framework for the Metaverse.” MathewBall.vc. Accessed 1 March, 2022. https://www.matthewball.vc/all/forwardtothemetaverseprimer Bradshaw, Tim and Hannah Murphy. 2021. “Facebook bets on virtual reality ‘metaverse’”. July 30. Financial Times. Accessed 1 March, 2022. https://www.ft.com/content/2b985dc5-3995-4ac3-a6f6-6ec0a0893c0b Epic Games. 2022. “MetaHuman Creator: High-fidelity digital humans made easy.” Unreal Engine. Accessed 1 March, 2022. https://www.unrealengine.com/en- US/metahuman-creator Gaboury, Jacob. 2021. Image Objects: An Archaeology of Computer Graphics. Cambridge Mass.: MIT Press. Gray, Mary L, Suri, Siddharth. 2019. Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass. Boston: Eamon Dolan Books. Lowe, Lisa. 2015. The Intimacies of Four Continents. Durham: Duke University Press. Phillips, Amanda. 2020. Gamer Trouble. New York: New York University Press. Srauy, Sam, and John Cheney-Lippold. 2019. “Realism in FIFA? How Social Realism Enabled Platformed Racism in a Video Game”. First Monday 24 (6). https://doi.org/10.5210/fm.v24i6.10091. Star, Susan Leigh, Strauss, Anselm. 1999. “Layers of Silence, Arenas of Voice: The Ecology of Visible and Invisible Work.” Computer Supported Cooperative Work (CSCW) 8: 9–30. Unreal Engine. 2021. “Digital Humans.” Unreal Engine 4.27 Documentation. Accessed 1 March, 2022. https://docs.unrealengine.com/4.27/en- US/Resources/Showcases/DigitalHumans Werning, Stefan. 2021. Making Games: The Politics and Poetics of Game Creation Tools. Cambridge Mass.: The MIT Press.