AoIR2018-submission Napoli Updated Abstract Revised Selected Papers of #AoIR2018: The 19th Annual Conference of the Association of Internet Researchers Montréal, Canada / 10-13 October 2018 HOW IS SOCIAL MEDIA GATEKEEPING DIFFERENT? A MULTI- PLATFORM COMPARATIVE ANALYSIS OF THE NEW YORK TIMES Peter Andringa Duke University David Duquette Duke University Deborah L. Dwyer University of North Carolina at Chapel Hill Philip M. Napoli Duke University Petra Ronald Microsoft Research Introduction News audiences are fragmented across different media platforms (Pew Research Center, 2016). Consequently, individual news organizations simultaneously disseminate their content across different media. Each of these media has different user bases, interface characteristics, and distribution systems. Given these substantial differences, the dynamics of the gatekeeping process – and the news values that guide this process – vary across different media technologies/platforms. As audience attention migrates from older to newer platforms (such as social media), it is important that we understand how the nature of the news that is disseminated – and thus consumed – may be different from the news disseminated through more traditional means. This study addresses these issues through a comparative gatekeeping analysis of the New York Times. For this case study, two month’s worth of Times front page, home page, and Facebook page story output was comparatively analyzed across dimensions such as story quantity, story duplication, hard versus soft news, and content diversity, in Suggested Citation (APA): Andringa,P., Duquette,D., Dwyer,D, Napoli,P., Ronald,P. (2018, October 10-13 Hatred Of/And Democracy: How Is Social Media Gatekeeping Different? A Multi- Platform Comparative Analysis Of The New York Times. Paper presented at AoIR 2018: The 19th Annual Conference of the Association of Internet Researchers. Montréal, Canada: AoIR. Retrieved from http://spir.aoir.org. an effort to determine how the nature of the news that is prioritized for news consumers differs between the social media context and older contexts. The Times provides an interesting cross-media comparison. For its decision-making regarding which of the over 300 stories produced each day to post to Facebook, the Times utilizes a self-developed bot called Blossom, which “predicts how articles or blogposts will do on social and also suggests which stories editors should promote by drawing from enormous stores of data, including information on story content and performance metrics such as Facebook post engagement” (Wang, 2015; Ellett, 2017). Journalism scholars, critics, and industry professionals have raised concerns about the diffusion of algorithmic decision-making into various aspects of the production, dissemination, and consumption of journalism (Bell & Owen, 2017). However, largely absent from these discussions have been comparative empirical analyses that examine if/how algorithmically assisted editorial outcomes differ from traditional, human- generated editorial outcomes. Research Questions RQ1: To what extent do the number of stories displayed differ across the Times’ front page, home page, and Facebook page? RQ2: To what extent is there story duplication across the Times’ front page, home page, and Facebook page? RQ3: How do the Times front page, home page, and Facebook page differ in terms of their ratios of hard to soft news? RQ4: How does story type diversity differ across the Times’ front page, home page, and Facebook page? Method Two month’s worth of story metadata was gathered from the Times’ and Facebook APIs in the spring of 2018. Only stories appearing on the front page, home page, or the main Times Facebook page were included in the analysis, under the logic that each of these contexts represents a mechanism to showcase the most relevant news of the day. For each story, the metadata includes the section in which the story appears in the print/online version of the paper, as well as story abstracts. These story section categories serve as the basis for the diversity analysis. Story abstracts were content analyzed by a trained coder using established criteria for categorizing individual stories as either hard or soft news. Intercoder reliability, utilziing Krippendorff’s Alpha, was .86. Results RQ1: Number of Stories Table 1: Number of Stories by Platform Platform Stories Avg. Stories/Day Front Page 1085 18.08 Home Page 2162 36.03 Facebook Page 3152 52.53 6399 RQ2: Story Duplication Table 2: Story Duplication Platform Percentage (N=4332) Front Page Only 5.79 Home Page Only 19.55 Facebook Page Only 36.66 Front Page and Home Page 1.89 Front Page and Facebook Page 7.64 Home Page and Facebook Page 18.74 Front Page, Home Page, and Facebook Page 9.72 RQ3: Hard/Soft News Table 3: Hard Versus Soft News Platform Hard News (%) Soft News (%) Front Page (n=1085) 80.97ac 19.03 Home Page (n=2139*) 74.57bc 25.43 Facebook Page (n=3219*) 74.87ab 25.13 χ2 = 19.1747 (p < .01) * Only individual stories were coded as hard/soft - articles like daily briefings and news quizzes are not included in the above numbers. a t = 4.10, p < .01. b t = .25, p > .05. c t = 4.07, p < .01. RQ4: Story Diversity Table 4: Top Four Story Type Concentration Ratios Platform Story Category Share (%) Front Page (n=1085) U.S. 27.94 World 19.30 Business 10.94 New York 9.38 Total: 67.56ab Home Page (n=2192) U.S. 20.48 World 13.32 Opinion 11.82 Business 7.44 Total: 53.06ac Facebook Page (n=3246) U.S. 22.40 World 17.71 Business 7.64 Opinion 6.87 Total: 54.62bc a t = 7.92; p < .01. b t = 7.48; p < .01. c t = 1.13; p > 05. Table 5: Story Type Diversity Across Platforms Platform HHI Front Page 1360.78 Home Page 791.94 Facebook Page 920.98 Conclusion To summarize, the New York Times showcases almost as many stories per day via Facebook as it does on its front page and home page combined. From a duplication standpoint, less than ten percent of the stories analyzed appear across all three platforms, indicating that, as gatekeeping mechanisms for showcasing the most newsworthy content, these platforms are operating somewhat differently in terms of individual story selection. From a hard/soft news standpoint, the Times home page and Facebook page were equivalent in maintaining a proportion of about 75% hard news; compared with about 80% for the front page. These results contradict the notion (at least within the context of the Times) that social media-disseminated news is likely to have a softer orientation than news showcased on other platforms. The degree of consistency across platforms suggests a fairly uniform commitment to hard news, even if the specific stories showcased differ across platforms. Finally, in terms of story type diversity, the Times home page exhibits the greatest story type diversity, followed closely by the Facebook page. Far less story type diversity was exhibited on the Times front page, suggesting that a narrower set of news values is being employed there in terms of the types of stories, subject matter-wise, that merit showcasing. That being said, the most prominent story types are fairly consistent across platforms. Going forward, it would be useful to build on this case study of a single news organization and conduct an analysis that applies this methodology to a large sample of publications. Such an expansion (if it were extensive enough) would allow for the investigation of whether outlet characteristics (organizational size, revenue model, ownership characteristics, etc.) bear a significant relationship to any of the cross- platform content similarities and differences investigated here. References Bell, E., & Owen, T. (2017). The platform press: How Silicon Valley reengineered journalism. Retrieved from https://www.cjr.org/tow_center_reports/platform- press-how-silicon-valley-reengineered-journalism.php Ellett, J. (2017, July 27). New AI-based tools are transforming social media marketing. Forbes. Retrieved from https://www.forbes.com/sites/johnellett/2017/07/27/new- ai-based-tools-are-transforming-social-media-marketing/#459b0c8169a2 Pew Research Center (2016). The modern news consumer. Retrieved from http://www.journalism.org/2016/07/07/the-modern-news-consumer/ Wang, S. (2015, August 13). The New York Times built a slack bot to help decide which stories to post to social media. Nieman Lab. Retrieved from http://www.niemanlab.org/2015/08/the-new-york-times-built-a-slack-bot-to-help- decide-which-stories-to-post-to-social-media/