key: cord-304183-zv3s7cjq authors: thirunavukarasu, arun james title: evaluating the mainstream impact of ophthalmological research with google trends date: 2020-11-01 journal: eye (lond) doi: 10.1038/s41433-020-01257-4 sha: doc_id: 304183 cord_uid: zv3s7cjq nan google interest in ophthalmological conditions was greatest between 2004 and 2005, falling to a minimum around 2011 with a subsequent rise to the present day interrupted by a sharp drop in march 2020 ( fig. 1) , likely due to disproportionate interest in the covid-19 pandemic. in contrast, pubmed publications tended to grow steadily, with a drop in 2020 ( fig. 1 ), attributable to both the pandemic and lack of four months' research relative to other years. a scatter plot relating the two variables ( fig. 2 ) indicated weak positive correlation overall, with most points lying outside the 95% confidence intervals of the best-fit trend-line. no correlation was apparent within search terms. correlation coefficients agreed with these deductions, with overall positive correlation (τ = 0.620, p = 1.35e−26) and no significant correlations (p > 0.05) within search terms. the overall correlation between google interest and pubmed publications indicates concordance between the interests of the scientific community and general public. however, lack of correlation within conditions suggests that ophthalmological research has little direct effect on laypeople's interests, which may instead be closer related to the prevalence of the respective conditions. correlation could also be masked by the lack of normalisation of pubmed data: ophthalmology may not have kept up with inflation observed across the scientific literature over time. further investigations could explore the effect on google interest of high impact publications, or publication-inspired news pieces. similar ophthalmological event analyses have previously been conducted, evaluating the effect of public health campaigns [3] , conjunctivitis epidemics [4] and bono developing glaucoma [5] . google trends as a surrogate marker of public awareness of diabetic retinopathy the use of google trends in health care research: a systematic review exploring the impact of public health campaigns for glaucoma and macular degeneration utilising google trends data in a new zealand setting google searches and detection of conjunctivitis epidemics worldwide still haven't found what i'm looking for… bono, google and glaucoma awareness conflict of interest the author declares no conflict of interest.publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. key: cord-252218-jrgl0x06 authors: heerfordt, c.; heerfordt, i. m. title: has there been an increased interest in smoking cessation during the first months of the covid-19 pandemic? a google trends study date: 2020-04-20 journal: public health doi: 10.1016/j.puhe.2020.04.012 sha: doc_id: 252218 cord_uid: jrgl0x06 [figure: see text] the city of wuhan in china became the centre of a pneumonia outbreak of unknown cause in december 2019. on 7 january 2020, a novel coronavirus, severe acute respiratory syndrome coronavirus 2, was isolated from the patients with pneumonia in wuhan. 1 the official name of the disease is coronavirus disease (covid-19) and common symptoms include fever, coughing and shortness of breath. most cases resolve spontaneously; however, some develop severe complications, including pulmonary oedema, severe pneumonia and acute respiratory distress syndrome. 2,3 on 10 april 2020, the total number of covid-19 cases reported worldwide was 1,563,857, including 95,044 deaths. the majority of deaths have been reported in italy (18,281), the us (16,690), spain (15,238), france (12,210) and the uk (7978). 11 unfortunately, the covid-19 pandemic is continuing to spread and there is an urgent need for measures to limit the harmfull effects of the virus. smokers are known to be more vulnerable to infectious diseases, including influenza and middle east respiratory syndrome-related coronavirus. 4, 5 smoking has also been found to be associated with negative progression and adverse outcomes for covid-19. 6 the current evidence comes from five chinese studies, which find smokers who are hospitalised with covid-19 have 2-9 times greater risk of serious covid-19 complications compared with non-smokers. 6 in the short term, smoking cessation leads to reduced respiratory symptoms and brochial hyperresponsiveness, and prevents unnecessary decline in lung function. 7 the covid-19 pandemic is having a major impact on the whole world and has gained huge public awareness. globally, millions of people search for health-related information online, which makes web search queries on google trends a valuable source of information on collective health trends. 8 the number of google searches on 'covid' and 'hand sanitizer' rose sharply in late february and march (see fig. 1 ). 9 this study aims to investigate the interest in quitting smoking during the first months of the covid-19 pandemic. as the interest in 'covid' and 'hand sanitizer' increaded rapidly in late february, we have examined the interest in smoking cessation from january to april 2020. data were collected from google trends (trends.google.com), which provides information on how many 'hits' different words had on a given day on google. this can be used as a measurement of public interest over time. the highest interest on a search query is quantified as 100 relative search volume (rsv), decreasing to 0 rsv indicating no interest. 8 we retrieved worldwide public query data for the following terms: 'quit smoking', 'smoking cessation', 'help quit smoking' and 'nicotine gum' between 9 january 2020 and 6 april 2020. we investigated whether there was an increased interest in quitting smoking in late febrary and march compared with the preceeding weeks. the google trends data for web search queries for the terms 'smoking cessation' and 'nicotine gum' from 9 january 2020 to 6 april 2020 are shown in fig. 1 . all search terms show stable interest over the selected time period; there was no tendency for increased interest in any of the key terms. outputs for the terms 'help quit smoking' and 'how do i quit smoking' are not shown in fig 1, but are available on trends.google.com and show the same stable trend. previous google trends studies have found increased numbers of seaches relating to smoking cessation in association with the launch of national smoking cessation programmes and changes in tobacco control policies. 10 we found no increase in the number of searches for smoking cessation on google in the first months of the covid-19 pandemic. this could indicate that there has been no actual increase in smoking cessation during the pandemic; however, this may change over the coming weeks and months, as the covid-19 pandemic is likely far from over. we hope that public health messages will focus on smoking cessation to improve lung health during this continued pandemic. smoking cessation campaigns are important as smokers are more vulnable to viral infections and lung diseases, and appear to have worse outcomes when hospitalised with covid-19 than non-smokers. 4-6 the novel coronavirus originating in wuhan, china: challenges for global health governance epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in wuhan, china: a descriptive study clinical characteristics of 138 hospitalized patients with novel coronavirus-infected pneumonia in wuhan, china mers transmission and risk factors: a systematic review cigarette smoking and infection covid-19 and smoking: a systematic review of the evidence the impact of smoking cessation on respiratory symptoms, lung function, airway hyperresponsiveness and inflammation google trends: a web-based tool for real-time surveillance of disease outbreaks more effective strategies are required to strengthen public awareness of covid-19: evidence from google trends. ssrn electron j [internet associations of the stoptober smoking cessation program with information seeking for smoking cessation: a google trends study. drug alcohol depend european centre for disease control and prevention. situation update worldwide, as of 10 key: cord-296821-qdhj9zj6 authors: uvais, nalakath a. title: interests in quitting smoking and alcohol during covid‐19 pandemic in india: a google trends study date: 2020-07-19 journal: psychiatry clin neurosci doi: 10.1111/pcn.13118 sha: doc_id: 296821 cord_uid: qdhj9zj6 nan it is a well established fact that, just like smoking, alcohol use is significantly associated with the risks for pneumonia [1, 2] . the negative effects of alcohol in the transmission and disease progression of viral infections are also well studied [3] . recent studies have highlighted the negative association of smoking and prognosis in patients with covid-19 [2, 4] . however, the effects of alcohol-related liver disease on prognosis of covid-19 is still under evaluation [5] . in response to a fake news that alcohol give protection from covid-19, who released detailed factsheet providing important information one should know about alcohol consumption and covid-19 [6]. this document advised readers to avoid alcohol altogether to protect immune system and stay sober to act quickly and make decisions with a clear head, for oneself and others in family and community. moreover, in an attempt to control the pandemic, indian government implemented strict physical distancing measures and advised the public to remain indoors and also banned the sale of alcohol and tobacco since march 25, this article is protected by copyright. all rights reserved. 2020. the ban was very strict and no tobacco or alcohol was available legally to buy either directly from stores or online, and it extended to more than two months. many mental health specialists and public health experts appeared in television and wrote in news papers to sensitise the public to utilise this period to quit smoking and alcohol. google trend analysis during the lockdown period showed a sharp increase in google searches on covid and disinfection measures in india. this study aims to investigate the interest in quitting smoking and alcohol during the lockdown period in india since march 25 to know the effectiveness of public awareness measures conducted regarding the negative aspects of smoking and alcohol during covid-19 pandemic. this methodology was adopted from a recent study with the understanding that a significant population of india search for health-related information online and google trend can inform regarding the collective health trends [7, 8] . as the interest in 'covid' and 'hand sanitizer' increaded rapidly during march, we have examined the interest in quitting smoking and alcohol from 2nd february to 29 april 2020. data were collected from google trends (trends.google.com), which provides information on how many 'hits' different words had on a given day on google, which can be used as a measurement of public interest over time [9] . the highest interest on a search query is quantified as 100 relative search volume (rsv), decreasing to 0 rsv indicating no interest. we retrieved public query data from india for the following terms: 'how to quit smoking' and 'how to quit alcohol' between 2 february 2020 and 1 may 2020. we investigated whether there was an increased interest in quitting smoking in late febrary, march, and april compared with the preceding weeks. this article is protected by copyright. all rights reserved. the interest in the search term 'how to quit smoking' showed significant increase on 9th march (90 rsv) ant the interest in the search term reached 100 rsv on 19th april (figure 1). the interest in the search term 'how to quit alcohol' showed significant increase on 11th february (100 rsv). however, the interest for both the search terms was not stable over the study period (figure 1). our study results showed no consistent increase in the number of searches for quitting smoking or quitting alcohol on google during the study period (february to may). a recent study analysing the google trend regarding smoking cessation searches worldwide during the early months of the covid-19 outbreak (9 january 2020 and 6 april 2020) also failed to show a tendency for increased interest in any of the key terms related to smoking cessation ('quit smoking', 'smoking cessation', 'help quit smoking' and 'nicotine gum') [8] . however, another study from netherlands showed a significant increase in rsv one to four weeks after the introduction of the smoking ban in restaurants and bars in 2008, and also after the introduction of smoking cessation support in 2011 [10] . our study results may indicate that there has been no significant increased interest in quitting smoking and alcohol, at least among the indian population who use online resources for health-related information. our results further highlight the need for continuing public health efforts to sensitise indian public regarding the negative effects of smoking and alcohol during covid-19 pandemic. however, our study results were preliminary, and further research is needed to know the long-term trend and compare it to the results of other studies. disclosure statement: this article is protected by copyright. all rights reserved. the number of google searches on 'covid' and 'hand sanitizer' increased sharply worldwide in march. on the other hand, the interest in 'how to quit smoking' and 'how to quit alcohol' showed no consistent changes for increased interest. alcohol use as a risk factor in infections and healing: a clinician's perspective covid-19 and smoking: a systematic review of the evidence alcohol's role in hiv transmission and disease progression clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus-infected pneumonia in wuhan, china implications of covid-19 for patients with pre-existing digestive diseases use of internet for accessing healthcare information among patients in an outpatient department of a tertiary care center has there been an increased interest in smokingcessation during the first months of the covid-19 pandemic? a google trends study google trends: a web-based tool for real-time surveillance of disease outbreaks effect of tobacco control policies on information seeking for smoking cessation in the netherlands: a google trends study this article is protected by copyright. all rights reserved web search queries for the terms 'covid', 'hand sanitizer' , 'how to quit smoking' and 'how to quit alcohol'. the number of google searches on 'covid' and 'hand sanitizer' increased sharply worldwide in march. on the other hand, the interest in 'how to quit smoking' and 'how to quit alcohol key: cord-302758-i5pe61h1 authors: pier, matthew m.; pasick, luke j.; benito, daniel a.; alnouri, ghiath; sataloff, robert t. title: otolaryngology-related google search trends during the covid-19 pandemic date: 2020-06-19 journal: am j otolaryngol doi: 10.1016/j.amjoto.2020.102615 sha: doc_id: 302758 cord_uid: i5pe61h1 objective: to assess trends of google search queries for symptoms and complaints encountered commonly in otolaryngology practices during the coronavirus disease 2019 (covid-19) pandemic when in-person care has been limited. materials and methods: in this cross-sectional study, data on google search queries in the united states for 30 otolaryngology-related terms were obtained from google trends. the means of relative search volume from the covid-19 period (march 29, 2020 through may 16, 2020) were compared to similar periods from 2016 to 2019 using a t-test of two independent samples. results: in total, 16.6% of search terms had significant increases in relative search volume during the covid-19 period, with the largest percentage increase for “can't smell” (124.4%, p = .006), followed by “allergies” (30.3%, p = .03), “voice pain” (26.1%, p = .008), and “ears ringing” (19.0%, p < .001). of all search terms, 26.7% had significant decreases in relative search volume, including the largest percentage decrease for “laryngitis” (59.8%, p < .001), followed by “thyroid nodule” (54.4%, p < .001), “thyroid cancer” (45.6%, p < .001), and “ent” (34.9%, p < .001). conclusion: this study demonstrates that google search activity for many otolaryngology-related terms during the covid-19 pandemic has increased or decreased significantly as compared to previous years. with reduced access to in-office otolaryngology care in the united states during the covid-19 pandemic, these are important considerations for otolaryngology practices to meet the needs of patients who lack access to care. the world health organization declared the coronavirus disease 2019 (covid-19) a global pandemic on march 11, 2020 [1] . in the united states (u.s.), federal guidelines for "social distancing" followed shortly thereafter, and by march 30, 30 states had officially announced "stay-at-home" orders, and 12 additional states issued similar orders during the following week [ [2] , [3] ]. outpatient medical visits, including those in otolaryngology practices, have been reduced drastically secondary to these measures in an effort to protect patients and providers, as well as preserve personal protective equipment [4] . changes in regulations have prompted providers to offer more telehealth opportunities, and some otolaryngology practices have increased utilization of this method of care delivery [ [4] , [5] , [6] ]. for patients unable to access in-person or telehealth care, it is plausible that they might increasingly utilize internet search engines such as google search (google inc., mountain view, california) to obtain information about symptoms, diagnoses, or treatments. during this unprecedented situation, it is important to understand the alternative ways in which patients are obtaining medical information in order to develop strategies within the otolaryngology community for meeting their needs. google trends (gt) is a free and publicly available tool that provides information on geospatial and temporal patterns in search volumes for user-specified terms [7] . gt has contributed to the emerging field of "infodemiology", showing utility in assessing outbreaks of influenza-like illness or providing unique insights into human behavior [ [8] , [9] , [10] ]. this tool also has been utilized to examine relationships of otolaryngology-related search terms with environmental factors [11] , [12] . this study aims to assess trends within the u.s. for google search queries of symptoms and complaints encountered commonly in otolaryngology practices comparing the time of covid-19 pandemic with similar time periods in previous years. gt determines the proportion of each search term among an anonymized sample of all search requests performed using google search [13] . repeated searches by the same individual over a short period of time are eliminated. to account for geographical differences in total search volumes, gt normalizes each data point by dividing by the total searches within the time range and region. relative search volume (rsv) is presented on a scale from 0 to 100 based on a term's proportion to all other searches, with the query's peak for the time range and geographic region set at 100 and all other data points divided by that peak value. for example, a value of 100 is the peak popularity for the term, while a value of 50 means that the term is half as popular. table 1 includes the 30 search terms selected by the authors. these included a range of otolaryngology-related symptoms and complaints in technical and non-technical terminology as well as variations on a term to capture search activity more broadly. internet search activity data were obtained and downloaded from the gt website (https://trends.google.com/). each term search was performed with the following parameters: "united states" region; "1/1/2016 through 5/16/2020" time range; "all categories" for category; "web search" type of search. our aim was to investigate search volume coinciding with "stay-at-home" orders issued by the majority of states in the u.s. by march 30, 2020. 3 rsv data from the periods in 2016-2019 were pooled, the means calculated, and the rsv means from the corresponding covid-19 timeframe in 2020 were calculated. comparison of the rsv means between these time periods was performed using a t-test of two independent samples assuming unequal variances with two-sided p-value<.05 considered statistically significant. all computations were conducted in microsoft excel 2017 (microsoft corporation, redmond, washington). comparisons of the means of rsv and percentage change for respective search terms are described in table 2 . of the 5 (16.6%) search terms with statistically significant increases in rsv during the covid-19 period, the largest percentage increase was for "can't smell" three pairs of non-technical and technical terms were examined: "ears ringing" and "tinnitus", "hoarseness" and "dysphonia", and "trouble swallowing" and "dysphagia". "ears ringing" demonstrated a significant 19.0% increase in rsv (μ 1 p=.60), while "dysphagia" decreased significantly by 8.5% (μ 1 -μ 2 =-6.5 [95% ci -11.1 to -1.8]; p=.02). during the covid-19 pandemic when "stay-at-home" orders and "social distancing" guidelines drastically affected everyday life as well as otolaryngology practices in the u.s., we present evidence that internet search activity for many otolaryngology-related terms has changed significantly. of the 5 search terms with significant increases in rsv, "can't smell" demonstrated the greatest increase. a likely contributing factor to this increase was the inclusion of olfactory dysfunction as a possible symptom of covid-19. by mid-march 2020, the american academy of otolaryngology-head and neck surgery recommended including this symptom when screening for possible covid-19 [14] . meta-analysis later indicated a 52% prevalence of olfactory dysfunction among infected patients [15] . this was discussed widely in the media, j o u r n a l p r e -p r o o f 7 which likely contributed to the 124.4% increase in google searches for "can't smell" seen in this study [16] . the four additional search terms with significant increases in rsv were "allergies", "voice pain, "ears ringing" and "ear pain". in attempting to understand this trend, it is interesting that all of these terms are non-technical and could be associated with patients being unable to attend appointments with otolaryngologists. as a quantitative example of appointment reductions in an otolaryngology practice, kasle et al. reported a roughly 80% reduction in completed appointments from mid-march 2020 to mid-april 2020 compared with the corresponding period in 2019 [4] . similarly, emergency departments have seen 80% reductions in the number of otolaryngology-related consultations during the covid-19 pandemic [17] . it is possible that the increases in search queries for these four non-technical terms could be attributed partially to patients resorting to internet searches in lieu of medical consultation to obtain information about symptoms or treatment. however, this association is speculative and warrants further investigations such as correlating gt searches for "allergies" with regional pollen concentrations [18] . nonetheless, this finding highlights the increasing use of the internet for certain nontechnical symptoms during the covid-19 period. with the marked variability in the quality of internet information on treatments for common otolaryngologic problems, this highlights the need for improvements to the web-based information available to patients [19] . of the 30 terms investigated, 26.5% demonstrated significant decreases in rsv during the covid-19 period. in assessing the possible factors contributing to these trends, it may be important to recognize the 34.4% reduction in rsv for "ent" as the context for the decreases seen in the other terms. this decrease in "ent" queries could reflect the reduction in referrals to otolaryngologists and the widespread cancellations of elective surgeries [20] . previous studies have shown that patients utilize more internet search terms after receiving a diagnosis as j o u r n a l p r e -p r o o f 8 compared to before it has been given [21] . with fewer patients accessing otolaryngologists to receive diagnoses such as thyroid cancer or dysphonia, a decreased post-diagnosis search phenomenon could explain the reductions in certain search queries. this is further supported by the findings for the three pairs of non-technical and technical terms compared in this study. the only significant rsv increase for these was the non-technical term "ears ringing", while the only significant rsv decreases occurred with the technical terms "dysphonia" and "dysphagia." rsv reductions for these terms, as well as "thyroid nodule", could provide insight into patients' diagnostic needs that may have been addressed insufficiently during this period. prompt attention to this matter may be warranted, as a portion of these diagnoses may represent underlying cancer and delays in care for head and neck cancer are associated with decreased survival [22] . period to compensate for the reduction of in-office encounters, but this degree of technological adaptation has not been seen nationally [ [4] , [23] ]. there is a substantial amount of literature addressing the unique challenges to implementing telehealth in otolaryngology practices, and these gt findings could be incorporated into these strategies to meet patients' needs both during the covid-19 pandemic and after it has resolved [ [24] , [25] , [26] ]. looking beyond the ongoing pandemic, gt represents a potentially powerful source of insight into population-level searches for medical information that could guide timing of public health policies and improve understanding of regional differences in otolaryngology-related patient needs. this report has limitations. the selected terms that were investigated represent a subjective bias toward terms that the authors suspected to be searched commonly on the internet. gt does not provide quantitative information on search terms. therefore, the observed changes in this study cannot be quantified to determine actual numbers of searches. data are reported j o u r n a l p r e -p r o o f 9 from a population sample determined by gt, and thus may not represent accurately the entire population of the region under investigation. additionally, there is inherent bias when using gt data as they are generated by a population that is literate, technologically competent, has access to the internet, and has selected google as a search engine. the underlying motivations to perform a google search vary widely and could include obtaining information on symptoms experienced by individuals, responding to media coverage, seeking information for academic interests, or many other motivations. these motivations cannot be discerned directly from gt data, which limits the interpretation of the information. nevertheless, the trends suggest that patients are responding to covid-19-related restrictions in otolaryngology care by changing their patterns of self-education through the internet. the covid-19 pandemic has challenged the ability of otolaryngologists to provide care to many patients in the u.s. this study demonstrates that google search activity for many otolaryngology-related terms during this period has increased or decreased significantly as compared to previous years. these trends may suggest unmet needs of patients that otolaryngology practices should consider and attempt to address through expanded telehealth utilization. it also suggests that otolaryngologists may need to be prepared to address the information and misinformation that patients have acquired, which is likely to affect the substance of future in-person or telehealth encounters. additional investigations are needed to better understand how patients utilize the internet to obtain medical information, to improve the quality of available web resources, and to determine the effects of such self-education on patients' future behaviors in caring for their otolaryngologic disorders. this research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. relative search volume "can't smell" "allergies" pandemic march: covid-19's first wave circumnavigates the globe coronavirus guidelines for america | the white house. the white house when state stay-at-home orders due to coronavirus went into effect outpatient otolaryngology in the era of covid-19: a data-driven analysis of practice patterns. otolaryngol head neck surg covid-19 and health care's digital revolution telehealth opportunities for the otolaryngologist: a silver lining during the covid-19 pandemic. otolaryngol head neck surg google trends. google trends assessing the methods, tools, and statistical approaches in google trends research: systematic review correlation between national influenza surveillance data and google trends in south korea forecasting type-specific seasonal influenza after 26 weeks in the united states using influenza activities in other countries doctor google: correlating internet search trends for epistaxis with metropolitan climates effect of environmental factors on internet searches related to sinusitis faq about google trends data -trends help anosmia, hyposmia, and dysgeusia symptoms of coronavirus disease. american academy of otolaryngology-head and neck surgery the prevalence of olfactory and gustatory dysfunction in covid-19 patients: a systematic review and meta-analysis otolaryngol head neck surg doctors warn an inability to smell could be a symptom of covid-19 covid-19: what happened to all of the otolaryngology emergencies? google trends and pollen concentrations in allergy and airway diseases in france how good is google? the quality of otolaryngology information on the internet evolving management of covid-19: a multiinstitutional otolaryngology perspective. otolaryngol head neck surg online information seeking behaviors of breast cancer patients before and after diagnosis: from website discovery to improving website information quantitative survival impact of composite treatment delays in head and neck cancer covid-19 transforms health care through telemedicine: evidence from the field smartphone-enabled wireless otoscope-assisted online telemedicine during the covid-19 outbreak embracing telemedicine into your otolaryngology practice amid the covid-19 crisis: an invited commentary diagnosis and treatment of voice disorders weekly google search results for "laryngitis" and "thyroid nodule" a a time periods analyzed are in bold, which include the covid-19 period relative search volume "laryngitis key: cord-262310-z0m6uuzf authors: effenberger, maria; kronbichler, andreas; shin, jae il; mayer, gert; tilg, herbert; perco, paul title: association of the covid-19 pandemic with internet search volumes: a google trendstm analysis date: 2020-04-17 journal: int j infect dis doi: 10.1016/j.ijid.2020.04.033 sha: doc_id: 262310 cord_uid: z0m6uuzf abstract objectives to assess the association of public interest in coronavirus infections with the actual number of infected cases for selected countries across the globe. methods we performed a google trendstm search for “coronavirus” and compared relative search volumes (rsv) indices to the number of reported covid-19 cases by the european center for disease control (ecdc) using time-lag correlation analysis. results worldwide public interest in coronavirus reached its first peak end of january when numbers of newly infected patients started to increase exponentially in china. the worldwide google trendstm index reached its peak on the 12th of march 2020 at a time when numbers of infected patients started to increase in europe and covid-19 was declared a pandemic. at this time the general interest in china but also the republic of korea has already been significantly decreased as compared to end of january. correlations between rsv indices and number of new covid-19 cases were observed across all investigated countries with highest correlations observed with a time lag of -11.5 days, i.e. highest interest in coronavirus observed 11.5 days before the peak of newly infected cases. this pattern was very consistent across european countries but also holds true for the us. in brazil and australia, highest correlations were observed with a time lag of -7 days. in egypt the highest correlation is given with a time lag of 0, potentially indicating that in this country, numbers of newly infected patients will increase exponentially within the course of april. conclusions public interest indicated by rsv indices can help to monitor the progression of an outbreak such as the current covid-19 pandemic. public interest is on average highest 11.5 days before the peak of newly infected cases. a novel coronavirus, the acute respiratory syndrome coronavirus 2 (sars-cov-2), causes a new disease named corona virus disease 2019 . it was first detected in december 2019 in wuhan (hubei, china) (wang et al., 2020) . due to a high virulence and a high proportion of asymptomatic cases, the outbreak spreads all over the world. on april 5 th 2020 the world health organization (who) reported 1 133 758 confirmed cases. today, a cumulative mortality rate of 5.5 % (62 784) has been reported. the internet is increasingly used as a source of health care information. infodemiology and infoveillance are essential public health informatics methods which are used to analyze search behavior on the internet. infodemiology is defined as "science of distribution and determinants of information in an electronic medium, specifically the internet, or in a population, with the ultimate aim to inform public health and public policy", while the primary aim of infoveillance is surveillance (eysenbach, 2009) . infodemiology and infoveillance of epidemiological data are important to increase situational awareness and make suitable interventions (rivers et al., 2019) . the analysis of relative internet search volumes (rsv) gives information on the extent of public attention (arora et al., 2019 , kaleem et al., 2019 , ling and lee, 2016 with google trends tm being one of the most widely used tools for this purpose. rsv are used for real-time analyses for transmissibility, severity, and natural history of an emerging pathogen, as observed with severe acute respiratory syndrome (sars), the 2009 influenza pandemic, and ebola (chowell et al., 2009 , cleaton et al., 2016 . the analyses of confirmed cases are particularly useful to infer key page 5 of 18 j o u r n a l p r e -p r o o f 5 epidemiological parameters, such as the incubation and infectious periods and ongoing outbreaks or an outbreak probability. in addition, google trends tm data might be used to forecast an increase in infected cases. a linear time series pattern with official dengue reports, indicating a potential use to monitor public interest before an increase of cases and during the outbreak (husnayain et al., 2019) . beside infectious diseases, google trends tm have been successfully used to forecast the suicide risk increase (barros et al., 2019) . in this study, we investigated the public interest in covid-19 since december 31 st 2019 comparing google trends™ data to data of newly infected covid-19 cases. retrieving outbreak and confirmed cases numbers from the who data on confirmed covid-19 cases were retrieved on the 5 th of march from the european center for disease control (ecdc) for the time from the 31 st of december 2019 until the 1 st of april 2020 (https://www.ecdc.europa.eu/en/publicationsdata/download-todays-data-geographic-distribution-covid-19-cases-worldwide). worldwide data were retrieved as well as data for the following countries, namely china, republic of korea, japan, iran, italy, austria, germany, the united kingdom (uk), the united states (us), egypt, australia, and brazil. retrieving google trends tm data on covid-19 the google trends tm tool was used to retrieve data on internet user search activities in the context of covid-19. google trends tm enables researchers to study trends and patterns of google tm search queries (arora et al., 2019) . it was implemented in j o u r n a l p r e -p r o o f 6 trends tm expresses the absolute number of searches relative to the total number of searches over the defined period of interest (arora et al., 2019) . the retrieved google trends tm index ranges from 0 to 100, with 100 being the highest relative search term activity for the specified search query in the time period of interest. further information on google trends tm can be found on the respective help page worldwide interest in coronavirus started on january 20 th and reached its first peak on january 31 st , a few days after the word was spread on the outbreak in wuhan, china. the increasing numbers of cases over the globe prompted the who to declare the coronavirus outbreak as a pandemic on march 11 th , leading to an increase in public interest currently peaking on march 12 th 2020 ( figure 1 ). the data on newly confirmed cases, overall confirmed cases, and overall death worldwide as 7 well as for the afore-mentioned countries under study are summarized in table 1. there are two peaks, one sharp increase in numbers when cases were counted based on clinical diagnosis and not from a confirmatory laboratory test in china and the other peak on march 16 th due to cases around the globe. the worldwide initial peak associates with a strong increase of confirmed cases in china. in china, a maximum of google trends tm rsv was observed at the end of january with a 5.47-fold increase of cases between january 24 th and january 28 th . afterwards, with rigorous measures the relative increase in new cases was slower, and a decrease of new cases was firstly reported to the who on february the 6 th , with the exception of a sharp increase as mentioned above. the rsv trend followed a similar path, with a steady number of search enquiries around 25% of the maximal interest during the last weeks. (figure 2 ). correlation analysis indicates highest public interest in covid-19 on average around 11.5 days before the maximum of newly infected cases was reported ( figure 3 ). in countries with proximity to china such as the republic of korea or japan a high volume of search queries was observed during or closely after the peak was reached in china. a non-comparable smaller peak was observed in countries in the european union or the us (figure 2 ). in the republic of korea, a first google trends™ index peak was observed end of january only slightly shifted as compared to the peak in china with a second peak being observed on february 23 rd (figure 2 ). this second peak in korea proceeded the peak in newly infected cases by 7 days (figure 3 ). japan´s rsv started to increase on february 24 th , with a peak on february 27 th , also followed by an increase in confirmed covid-19 cases. in iran, the most affected country in the middle east, a strong increase of rsv could be observed on february 18 th with a peak between 20 th and 22 nd of february. the iranian increase of rsv was five days page 8 of 18 j o u r n a l p r e -p r o o f 8 before the first confirmed cases in iran, with also a strong association and prediction of the outbreak, which followed five to seven days later. egypt, the first country on the african continent with a confirmed covid-19 case, showed a small rsv peak during the outbreak in china. furthermore, the rsv started to steadily increase since february 20 th with an observed leap in interest on april 1 st . australia showed a similar pattern with an increase in rsv during the first outbreak in china, followed by a decrease afterwards and again an increase since february 23 rd , followed by increasing new covid-19 cases 10 days later (figure 2 ). in european countries, especially in italy, a small peak in the google trends tm analysis was found during the outbreak in china and a climax was found on february 23 rd 2020, a few days before the numbers of newly covid-19 started to increase exponentially. similar trends were observed in austria, germany and the uk with a delay of several days and a second peak, which was accompanied by an increase in numbers in the following days. the highest rsv peak was reached mid of march , which is in line with rigorous policies by the government regarding the rapid spread. the uk and australia show very similar patterns with highest correlations between rsv indices and newly diagnosed cases found with time lags of -12 and -7 days respectively ( figure 3 ). in the us, a steady increase of google tm search queries since february 27 th was observed followed by an outbreak since march 2 nd . the peak of search queries was march 3 rd a new increase in rsv is found in brazil, followed by increasing numbers of newly confirmed cases of covid-19 ( figure 2 ). in our study, we found a significant increase in rsv using google trends™ for covid-19 worldwide with a peak of rsvs around 11.5 days prior to the peak in newly diagnosed cases in different countries all over the world. as such, google trends™ can be used to associate and predict outbreaks worldwide and provides a valuable picture of the outbreak of covid-19 in real time. close monitoring and continued evolution of enhanced communication strategies is needed that provide general populations and vulnerable populations most at risk with actionable information for self-protection, including identification of symptoms (heymann et al., 2020) . the application of internet data in health care research, also known as infodemiology, is a promising new field and it may complement and extend the current data sources and foundations (mavragani and ochoa, 2019) . the attention to covid-19 increased days to weeks before the actual peak outbreak, not only worldwide, but also in most of the investigated countries in this study. this strongly supports our finding that the rsv is a useful tool to monitor local and global outbreaks of infectious diseases. the internet is the biggest platform for search engines and social media for real time data and outbreaks. rsv has been used before to detect outbreaks, like the recent severe influenza outbreak in 2009 (cook et al., 2011) . close monitoring and continued evolution of enhanced communication strategies is needed that provide general populations and vulnerable populations most at risk with actionable page 10 of 18 j o u r n a l p r e -p r o o f 10 information for self-protection, including identification of symptoms (heymann et al., 2020) . most countries and the who provide awareness -raising and educational programs on covid-19 via internet. the strong association between rsv and increasing outbreak numbers may be due to implementation of such programs in the different countries. the impact of web based research continuously grows since the past decade (jun et al., 2018) . google trends™ is the only unbiased approach including millions of users and has widely been used in health issues. public attention in different fields has been published recently (e.g. osteoarthritis, breast cancer or copd) (boehm et al., 2019 , jellison et al., 2018 , kaleem et al., 2019 . furthermore, infodemiology and google trends™ is used to generate awareness profiles and is a suitable substitute for classical data collection, such as surveys (jun et al., 2018) . far mostly, google trends™ is used to monitor disease control and awareness in cancer, hiv or stroke, but also in rare diseases like antiphospholipid syndrome or systemic lupus erythematosus (ling and lee, 2016 , mahroum et al., 2019 , sciascia and radin, 2017 , sciascia et al., 2018 . definitely, google trends™ can be used to detect success rates of awareness programs and predict infectious outbreaks worldwide (mclean et al., 2019 , patel et al., 2020 . there are also some potential limitations of this study. there is no information about the individual searches for the analyzed topics. the selections of spelling/terms might affect the results and conclusions. the importance of accuracy in defining the search queries is exemplified by searching google trends™ for the topic "pneumonia". pneumonia is associated with covid-19, although not specifically representing covid-19. thus, using the query "pneumonia" may be useful to analyze symptom-related curiosity, but does not sufficiently represent covid-19 outbreaks. the number of studies based on google trends™ is increasing, but so far there is no standardized procedure for data collection. more guidance by google™ should be warranted in order to assist researchers to establish an optimal search strategy (nuti et al., 2014) . despite the fact the google search is accessible worldwide, the use of different search tools in certain countries like for example baidu in china might lead to more accurate estimations of public interest. it was for example shown that a high baido search index (bsi) predicted dengue fever outbreaks in guangzhou and to a lesser degree in zhongshan, indicating that bsi might complement traditional dengue fever surveillance in china (liu et al., 2016) . in our study we decided to make use of data from one common framework. in conclusion, infodemiology and rsv provide a tool to anticipate covid-19 outbreaks and of other infectious diseases. information on public interst could be used to monitor the outbreak in northern european countries, africa or the americas. cases, with highest interest observed on average 11.5 days before the peak of newly reported covid-19 cases google trends: opportunities and limitations in health and health policy research the validity of google trends search volumes for behavioral forecasting of national suicide rates in ireland using google trends to investigate global copd awareness severe respiratory disease concurrent with the circulation of h1n1 influenza characterizing ebola transmission patterns based on internet news reports assessing google flu trends performance in the united states during the 2009 influenza virus a (h1n1) pandemic infodemiology and infoveillance: framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the internet technical advisory group for infectious h. covid-19: what is next for public health? correlation between google trends on dengue fever and national surveillance report in indonesia using google trends to assess global public interest in osteoarthritis ten years of research change using google trends: from the perspective of big data utilizations and applications google search trends in oncology and the impact of celebrity cancer awareness disease monitoring and health campaign evaluation using google search activities for hiv and aids, stroke, colorectal cancer, and marijuana use in canada: a retrospective observational study using baidu search index to predict dengue outbreak in china capturing public interest toward new tools for controlling human immunodeficiency virus (hiv) infection exploiting data from google trends google trends in infodemiology and infoveillance: methodology framework internet search query analysis can be used to demonstrate the rapidly increasing public awareness of palliative care in the usa the use of google trends in health care research: a systematic review success of prostate and testicular cancer awareness campaigns compared to breast cancer awareness month according to internet search volumes: a google trends analysis using "outbreak science" to strengthen the use of models during epidemics what can google and wikipedia can tell us about a disease? big data trends analysis in systemic lupus erythematosus infodemiology of antiphospholipid syndrome: merging informatics and epidemiology the authors declare no conflicts of interest. not applicable key: cord-330936-qf4q8yqq authors: kardeş, sinan; kuzu, ali suat; raiker, rahul; pakhchanian, haig; karagülle, mine title: public interest in rheumatic diseases and rheumatologist in the united states during the covid-19 pandemic: evidence from google trends date: 2020-10-18 journal: rheumatol int doi: 10.1007/s00296-020-04728-9 sha: doc_id: 330936 cord_uid: qf4q8yqq to evaluate the public interest in rheumatic diseases during the coronavirus disease 2019 (covid-19) pandemic. google trends was queried to analyze search trends in the united states for numerous rheumatic diseases and also the interest in a rheumatologist. three 8-week periods in 2020 ((march 15–may 9), (may 10–july 4), and (july 5–august 29)) were compared to similar periods of the prior 4 years (2016–2019). compared to a similar time period between 2016 and 2019, a significant decrease was found in the relative search volume for more than half of the search terms during the initial march 15–may 9, 2020 period. however, this trend appeared to reverse during the july 5–august 29, 2020 period where the relative volume for nearly half of the search terms were not statistically significant compared to similar periods of the prior 4 years. in addition, this period showed a significant increase in relative volume for the terms: axial spondyloarthritis, ankylosing spondylitis, psoriatic arthritis, rheumatoid arthritis, sjögren’s syndrome, antiphospholipid syndrome, scleroderma, kawasaki disease, anti-neutrophil cytoplasmic antibody (anca)-associated vasculitis, and rheumatologist. there was a significant decrease in relative search volume for many rheumatic diseases between march 15 and may 9, 2020 when compared to similar periods during the prior 4 years. however, the trends reversed after the initial period ended. there was an increase in relative search for the term “rheumatologist” between july and august 2020 suggesting the need for rheumatologists during the covid-19 pandemic. policymakers and healthcare providers should address the informational demands on rheumatic diseases and needs for rheumatologists by the general public during pandemics like covid-19. the recent pandemic of coronavirus disease 2019 has placed the rheumatologists and immunologists at the forefront of understanding the pathogenesis and management of this infection [1] [2] [3] [4] . after severe acute respiratory syndrome coronavirus 2 (sars-cov-2) cellular infection, the host immune response is triggered and generates proinflammatory cytokines and chemokines [5] . these molecules recruit monocytes, macrophages, and t cells which leads to create further inflammation that inevitably establishes a pro-inflammatory feedback loop [5] . a defective immune response aggressively generates pro-inflammatory cytokines, called cytokine storms, which can lead to multiple-organ dysfunction [5] . rheumatologists and immunologists have substantial expertise in the pathogenesis of the immune response, the impact of inflammation, and their treatments [1] [2] [3] [4] . several immunomodulatory and antiinflammatory drugs (e.g., hydroxychloroquine, colchicine, tocilizumab, anakinra) have been used for covid-19 when scientists are in search of a possible treatment for this novel virus and a resultant inflammatory response [1] [2] [3] [4] [6] [7] [8] . since rheumatologists have substantial knowledge in the benefits and adverse effects of these drugs as well, rheumatologists are playing a significant role in this pandemic. besides fighting covid-19, the rheumatologists also have an essential role in caring for their rheumatic patients who may express concerns about their potential increased risk of acquiring covid-19 infection due to their underlying disease itself and any immunosuppressive treatments they may be on [9] [10] [11] [12] [13] [14] [15] [16] . google trends has been demonstrated as a powerful tool in surveying outbreaks and monitoring public interest [17] [18] [19] [20] . it was used in previous outbreaks including influenza [21] [22] [23] and zika virus [24] . interestingly, for covid-19 pandemic, google trends was used to evaluate public interest in other medical disciplines including dermatology [25] , plastic surgery [26] , urology [27] and otolaryngology [28] . these studies highlight the integration of google trends to the field of medicine as a valuable epidemiologic data to monitor public interest. the public interest in rheumatic diseases and rheumatologist during the pandemic represents a knowledge gap. this information may be useful to determine whether the public has access to rheumatological-related information that may guide to policymakers and healthcare providers to address the informational demands and needs for rheumatologists during the covid-19 pandemic by the general public. therefore, the aim of this study was to use the google trends data to evaluate the public interest in rheumatic diseases and rheumatologist during the covid-19 pandemic. google trends provides the relative interest of google searches, on a range of 0-100 [29] . larger scores indicate greater relative interest. a wide range of search terms were determined to represent nearly all rheumatic diseases that patients might search on google (i.e., osteoarthritis, gout, pseudogout, calcium pyrophosphate crystal deposition (cppd), fibromyalgia, axial spondyloarthritis, ankylosing spondylitis, peripheral spondyloarthritis, psoriatic arthritis, reactive arthritis, septic arthritis, rheumatoid arthritis, sjögren's syndrome, systemic lupus erythematosus, antiphospholipid syndrome, scleroderma, polymyositis, dermatomyositis, relapsing polychondritis, familial mediterranean fever, tumor necrosis factor (tnf) receptor-associated periodic syndrome (traps), hyperimmunoglobulinemia d with periodic fever syndrome (hids), cryopyrin-associated periodic syndromes (caps), vasculitis, takayasu arteritis, giant cell arteritis, temporal arteritis, polyarteritis nodosa, kawasaki disease, polymyalgia rheumatica, anti-neutrophil cytoplasmic antibody (anca)associated vasculitis, granulomatosis with polyangiitis, and behçet's syndrome). in addition, the term "rheumatologist" was also analyzed. although some terms are used interchangeably for the disease (e.g., pseudogout and cppd), the study included all listed terms because the general public may not search for synonymous terms. each term was searched as presented except for calcium pyrophosphate crystal deposition (searched as cppd), sjögren's syndrome (as sjogren), systemic lupus erythematosus (as lupus), anca-associated vasculitis (as anca vasculitis), behçet's syndrome (searched as behcet). traps and hids could not be included because google trends did not provide data for these two diseases due to their low relative search volume. on september 6, 2020, these search terms were queried on google trends using filters of "united states", "01/01/2016-09/06/2020", and "all categories". the google trends weekly data were downloaded. three 8-week periods in 2020 ((march 15-may 9), (may 10-july 4) and (july 5-august 29)) were compared to similar periods of the prior 4 years (2016-2019) to evaluate both initial-and short-term interest. dates after march 15, 2020, when nationwide quarantine measures became implemented, were evaluated. generalized estimating equations with gamma distribution were used in comparisons. analyses were performed with spss 21.0. p values less than 0.05 were considered statistically significant. in the march 15-may 9, 2020 period, the relative search volume for 21 of the 32 search terms (i.e., osteoarthritis, gout, pseudogout, cppd, fibromyalgia, peripheral spondyloarthritis, reactive arthritis, septic arthritis, sjögren's syndrome, scleroderma, polymyositis, dermatomyositis, caps, vasculitis, takayasu arteritis, giant cell arteritis, temporal arteritis, polyarteritis nodosa, polymyalgia rheumatica, behçet's syndrome, and rheumatologist) statistically significantly decreased; however, the search term (i.e., psoriatic arthritis) statistically significantly increased compared to prior 4 years (table 1 ). in the may 10-july 4, 2020 period, the relative search volume for 11 of the 32 search terms (i.e., osteoarthritis, pseudogout, fibromyalgia, peripheral spondyloarthritis, systemic lupus erythematosus, scleroderma, polymyositis, caps, takayasu arteritis, temporal arteritis, and behçet's syndrome) statistically significantly decreased; however, four search terms (i.e. axial spondyloarthritis, ankylosing spondylitis, psoriatic arthritis, and giant cell arteritis) statistically significantly increased compared to prior 4 years ( table 1) . in the july 5-august 29, 2020 period, relative search volume of 7 of the 32 search terms (i.e., gout, fibromyalgia, peripheral spondyloarthritis, systemic lupus erythematosus, polymyositis, relapsing polychondritis, and takayasu arteritis) statistically significantly decreased; however, 10 search terms (i.e., axial spondyloarthritis, ankylosing spondylitis, psoriatic arthritis, rheumatoid arthritis, sjögren's syndrome, antiphospholipid syndrome, scleroderma, kawasaki disease, anca-associated vasculitis, and rheumatologist) statistically significantly increased compared to prior 4 years (table 1 ). the study found a statistically significant decrease in relative search volume of more than half of the search terms in the march 15-may 9, 2020 period. however, this trend appeared to reverse after this initial period. in the july 5-august 29, 2020 period, the relative volume of nearly half of the search terms were not significantly different when compared to similar periods of the prior 4 years; however, the terms axial spondyloarthritis, ankylosing spondylitis, psoriatic arthritis, rheumatoid arthritis, sjögren's syndrome, antiphospholipid syndrome, scleroderma, kawasaki disease, anca-associated vasculitis, and rheumatologist all statistically significantly increased compared to 4 years prior. previous studies evaluated the public interest in other medical disciplines during the covid-19 pandemic [25] [26] [27] [28] . even though direct comparison of the present study is limited by differences in study design, the medical discipline evaluated, and the time periods compared, the present findings are consistent with those of the previous studies [25] [26] [27] [28] . each study has shown a significant decrease in relative search volume for most of the search terms in the initial covid-19 pandemic period. shifting interest of patients from their rheumatic disease to covid-19 might be responsible for this finding in the initial period. in the july 5-august 29, 2020 period, the relative search volume for many of the search terms were either not statistically significant or significantly increased compared to similar periods of the prior 4 years. this finding may indicate that the interest in rheumatic diseases has gradually recovered shortly after the initial covid-19 period, and the general public has informational needs on a wide range of rheumatic diseases. interestingly, psoriatic arthritis was increased in all three periods evaluated in the study with an increase in relative search volume of axial spondyloarthritis, ankylosing spondylitis, rheumatoid arthritis, sjögren's syndrome, antiphospholipid syndrome, scleroderma, kawasaki disease, and anca-associated vasculitis in the july/august period. the gradual increased interest in rheumatic diseases might be due to the general public's lack of knowledge on potential risk that may exist between these diseases and immunosuppressive treatments with covid-19 [30, 31] . in a previous study using google trends, dey and zhao compared the relative search volume of kawasaki disease during february-may 2020 to similar period in 2019 [32] . they showed a similar level of public interest in kawasaki disease in february-march 2020 and 2019, and an increased interest after mid-april, 2020 [32] . the present study showed a similar increase in relative search of kawasaki disease in all three periods evaluated while reaching a statistical significance during the july 5-august 29, 2020 period. the findings from the previous study (i.e., increasing interest after mid-april) [32] suggest that the insignificant increase between march 15 and may 9, 2020 may be due to similar relative search volume between march and early april, 2020. on the other hand, this study also found an increased interest in relative search of antiphospholipid syndrome. news on kawasaki-like disease, and discussions on possible presence of antiphospholipid antibodies in covid-19 might contribute to the observed trends in the present study [33] [34] [35] [36] . it is interesting to note that there was no significant difference in relative search volume of the term "rheumatologist" between may 10 and july 4, 2020 compared to similar periods in 2016-2019; however, a statistically significant increase was seen in the july 5-august 29, 2020 period. this finding that indicates the increased need for rheumatologists during the covid-19 pandemic may be attributable to two factors: (a) patients with a rheumatic disease and/or receiving immunosuppressive treatments wanted to consult rheumatologists on their disease/treatments [30] ; (b) patients consult rheumatologists on properties of several anti-inflammatory and immunomodulatory drugs that are being used in the treatment of covid-19 because rheumatologists are experts in the use of these drugs [1, 7] . rheumatic patients may be at a higher risk of receiving covid infection with worse outcomes, as many of them tend to be susceptible from their underlying rheumatic disease or immunocompromised due to their treatment [9] [10] [11] [12] [13] [14] [15] [16] . thus, this pandemic has put great responsibility on the shoulders of the rheumatologists for caring for this vulnerable patient group. in addition, considerable knowledge of rheumatologists and immunologists on the management of patients with pathological immune responses coupled with their substantial clinical experiences with many of the drugs that are currently being tested as potential covid-19 treatment has led to rheumatologists and immunologists being placed right in the frontlines in facing covid-19 [1] [2] [3] [4] [6] [7] [8] . all these demonstrate that the rheumatologists and immunologists have an indispensable role during the covid-19 pandemic. a limitation of this study is that the google search engine was the only search engine used to conduct the analysis. however, because over 85% of internet users in the united states rely on google for information searches [37] , the data used for this study represent a large sample size of american search queries as a whole. moreover, the study evaluated the public interest only in initial and short term; therefore, a study examining the long-term interest during the pandemic is warranted in the future. in addition, the trends in covid-19 terms, which would allow a better interpretation of the present results, could not be included to avoid overlapping publication with previous studies examining the trends of public interest in these terms [38, 39] . another limitation of this study is that it does not provide information regarding the reasons for the observed trends. further research is necessary to address this important issue. in conclusion, there was a significant decrease in relative search volume for many rheumatic diseases between march 15 and may 9, 2020 when compared to similar periods during the prior 4 years. however, the trends reversed after the initial period ended. there was an increase in relative search for the term "rheumatologist" between july and august 2020 suggesting the need for rheumatologists during the covid-19 pandemic. policymakers and healthcare providers should address the informational demands on rheumatic diseases and needs for rheumatologists by the general public during pandemics like covid-19. rheumatologists' perspective on coronavirus disease 19 (covid-19) and potential therapeutic targets rheumatologists' practice during the coronavirus disease 2019 (covid-19) pandemic: a survey in egypt managing rheumatic diseases during covid-19 implications of covid-19 in pediatric rheumatology the trinity of covid-19: immunity, inflammation and intervention a systematic review of the prophylactic role of chloroquine and hydroxychloroquine in coronavirus disease-19 (covid-19) benefits and adverse effects of hydroxychloroquine, methotrexate and colchicine: searching for repurposable drug candidates perspectives of immune therapy in coronavirus disease incidence and prevalence of vaccine preventable infections in adult patients with autoimmune inflammatory rheumatic diseases (aiird): a systemic literature review informing the 2019 update of the eular recommendations for vaccination in adult patients with care for patients with rheumatic diseases during covid-19 pandemic: a position statement from aplar risk factors for hospital admissions related to covid-19 in patients with autoimmune inflammatory rheumatic diseases determinants of covid-19 disease severity in patients with underlying rheumatic disease aktay ayaz n (2020) does immunosuppressive treatment entail an additional risk for children with rheumatic diseases? a survey-based study in the era of covid-19 systemic autoimmune diseases, anti-rheumatic therapies, covid-19 infection risk and patient outcomes management of childhood-onset autoinflammatory diseases during the covid-19 pandemic coronavirus disease 2019 (covid-19) in autoimmune and inflammatory conditions: clinical characteristics of poor outcomes infodemiology and infoveillance: framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the internet google trends in infodemiology and infoveillance: methodology framework seasonal variation in the internet searches for gout: an ecological study seasonal variation in the internet searches for psoriasis google trends: a web-based tool for real-time surveillance of disease outbreaks accurate influenza monitoring and forecasting using novel internet data streams: a case study in the boston metropolis advances in nowcasting influenza-like illness rates using search query logs dynamic forecasting of zika epidemics using google trends analysis of dermatology-related search engine trends during the covid-19 pandemic: implications for patient demand for outpatient services and telehealth a google trends analysis of facial plastic surgery interest during the covid-19 pandemic public interest in elective urologic procedures in the covid-19 pandemic: a google trends analysis otolaryngology-related google search trends during the covid-19 pandemic the psychological state and changes in the routine of the patients with rheumatic diseases during the coronavirus disease (covid-19) outbreak in turkey: a web-based cross-sectional survey coronavirus disease 2019 (covid-19) in a patient with ankylosing spondylitis treated with secukinumab: a case-based review covid-19 and kawasaki disease: an analysis using google trends an outbreak of severe kawasaki-like disease at the italian epicentre of the sars-cov-2 epidemic: an observational cohort study presence of antiphospholipid antibodies in covid-19: case series study thrombosis in coronavirus disease 2019 (covid-19) through the prism of virchow's triad fluctuation of public interest in covid-19 in the united states: retrospective analysis of google trends search data on the predictability of covid-19 in usa: a google trends analysis author contributions conception or design of the study: sk; data collection, analysis, and/or interpretation: sk, ask, rr, hp, and mk; drafting the article: sk; critical revision of the article: sk, ask, rr, hp, and mk; final approval of the version of the article to be published: key: cord-289647-14ba5sro authors: panuganti, bharat a.; jafari, aria; macdonald, bridget; deconde, adam s. title: predicting covid-19 incidence using anosmia and other covid-19 symptomatology: preliminary analysis using google and twitter date: 2020-06-02 journal: otolaryngol head neck surg doi: 10.1177/0194599820932128 sha: doc_id: 289647 cord_uid: 14ba5sro objective: to determine the relative correlations of twitter and google search user trends concerning smell loss with daily coronavirus disease 2019 (covid-19) incidence in the united states, compared to other severe acute respiratory syndrome coronavirus 2 (sars-cov-2) symptoms. to describe the effect of mass media communications on twitter and google search user trends. study design: retrospective observational study. setting: united states. subjects and methods: google search and “tweet” frequency concerning covid-19, smell, and nonsmell symptoms of covid-19 generated between january 1 and april 8, 2020, were collected using google trends and crimson hexagon, respectively. spearman coefficients linking each of these user trends to covid-19 incidence were compared. correlations obtained after excluding a short timeframe (march 22 to march 24) corresponding to the publication of a widely read lay media publication reporting anosmia as a symptom of infection was performed for comparative analysis. results: google searches and tweets concerning all nonsmell symptoms (0.744 and 0.761, respectively) and covid-19 (0.899 and 0.848) are more strongly correlated with disease incidence than smell loss (0.564 and 0.539). twitter users tweeting about smell loss during the study period were more likely to be female (52%) than users tweeting about covid-19 more generally (47%). tweet and google search frequency pertaining to smell loss increased significantly (>2.5 standard deviations) following a widely read media publication linking smell loss and sars-cov-2 infection. conclusions: google search and tweet frequency regarding fever and shortness of breath are more robust indicators of covid-19 incidence than anosmia. mass media communications represent important confounders that should be considered in future analyses. results. google searches and tweets concerning all nonsmell symptoms (0.744 and 0.761, respectively) and covid-19 (0.899 and 0.848) are more strongly correlated with disease incidence than smell loss (0.564 and 0.539). twitter users tweeting about smell loss during the study period were more likely to be female (52%) than users tweeting about covid-19 more generally (47%). tweet and google search frequency pertaining to smell loss increased significantly (.2.5 standard deviations) following a widely read media publication linking smell loss and sars-cov-2 infection. conclusions. google search and tweet frequency regarding fever and shortness of breath are more robust indicators of covid-19 incidence than anosmia. mass media communications represent important confounders that should be considered in future analyses. t here has been considerable attention in the news media and medical literature regarding smell loss as a potential early manifestation of severe acute respiratory syndrome coronavirus 2 (sars-cov-2) infection. on march 22, 2020, for example, the new york times published a widely read article describing the mounting evidence of this association. 1 moreover, in a recently published study of 237 patients, wherein clinicians were surveyed regarding patient symptomatology using the coronavirus disease 2019 (covid-19) anosmia reporting tool (developed by the american academy of otolaryngology-head and neck surgery), 73% of patients reported anosmia prior to covid-19 diagnosis, and 26.6% reported smell loss as the heralding symptom. 2 with the current need for real-time epidemiological data, social media and internet user behavior may be uniquely suited to advise covid-19-related resource allocation and mitigation strategies. in a recent study by walker et al, 3 for example, google search frequency (google trends) pertaining to smell loss was correlated both with covid-19 disease and mortality. this study used the power of user-generated content in an electronic medium to study public health trends, also known as infodemiology. indeed, the relative distinctiveness of anosmia as a symptom, particularly compared to other covid-19 symptoms (ie, cough, fever, shortness of breath, and/or fatigue), may offer unique, temporally sensitive data related to sars-cov-2 infection and may be worthy of infodemiological investigation. however, conclusions regarding disease trends based on social media or internet search data are inferred, not definitive. a recent and well-known application of infodemiology, google flu trends, for example, sought to predict regional spikes in influenza but was shuttered after consistent overprojections. 4 the method of deriving the search terms and the media's influence on user behavior were cited reasons for failure and represent an important reason to exercise caution when using this type of data. 5 as such, although significant correlations between google searches pertaining to anosmia and covid-19 incidence have already been reported, our intention in the present study is to better understand the relative value of alternative infodemiological parameters (nonsmell symptoms, covid-19 searches and tweets) and platforms (twitter) in estimating covid-19 infection trajectory in the united states. twitter, as a social media platform, allows for real-time research into usergenerated opinions, feelings, and health status with concomitant demographic data. twitter, therefore, may serve as an important adjunct to google search user trends in infodemiological investigations. its use during the covid-19 pandemic, however, has not yet been reported. in this study, we detail our findings following a preliminary infodemiological exploration into covid-19 incidence and its correlation with multiple user trends in an online forum. specifically, we sought to (1) investigate twitter ''tweets'' as an alternative or adjunct to google trends to understand covid-19 incidence patterns, (2) elucidate the relative infodemiological value of google searches and tweets regarding smell loss compared to nonsmell covid-19 symptoms, and (3) understand the influence of news media on infodemiological trends related to smell loss. twitter data, including tweet frequency and inferred twitter user demographic information (sex and age), were cultivated using crimson hexagon, a web-based social media library and analysis platform that allows review of all publicly available tweets filterable by search terms, location, and date range. we used search terms concerning covid-19 (''covid,'' ''coronavirus,'' ''covid-19,'' ''sars-cov2,'' and ''covid19''), nonsmell symptoms of covid-19 (''shortness of breath,'' ''fatigue,'' ''cough,'' and ''fever''), and smell loss (''anosmia,'' ''loss of smell,'' ''reduced smell,'' ''decreased smell,'' ''lose your sense of smell,'' ''lost sense of smell,'' ''decreased sense of smell,'' ''decrease your sense of smell,'' ''decreased my sense of smell,'' ''reduce your sense of smell,'' ''reduced my sense of smell,'' ''reduced sense of smell,'' ''loss of sense of smell,'' ''loss of smell,'' ''hyposmia'') to collect twitterderived data concerning smell loss and covid-19. we excluded ''retweets,'' tweet replies, and tweets containing urls in the primary analysis to better understand user behavior and to avoid capturing more passive mass media communications. in crimson hexagon, twitter user information (age and sex) is derived from an algorithm accounting for variables including author interests, time since registration, and twitter users the user follows. the reported accuracy of inferred age assignment is 65% and is classified in 1 of 4 age groups (\18, 18-24, 24-34, and 35). 6 information pertaining google search trends was obtained using google trends, an open-access platform that provides normalized search frequency data (scale of 0-100). we extracted search frequency data concerning loss of smell (search terms included ''loss of smell,'' ''anosmia,'' ''lose smell,'' ''sense of smell,'' ''cannot smell,'' ''can't smell,'' and ''hyposmia''), covid-19 (same as twitter), nonsmell symptoms of covid-19 (same as twitter), and a commonly prescribed therapy for smell loss (''nasal irrigation'' and ''sinus rinse''). frequency of searches about dysgeusia (''dysgeusia,'' ''taste change,'' and ''taste loss'') was also obtained and correlated to covid-19 incidence. search terms enclosed in quotations automatically included queries with words before and after the first and last words, respectively, of the enclosed phrase. the specified time period for both crimson hexagon and google trends queries was january 1, 2020, through april 8, 2020, to encompass a ''control period'' for tweets and searches before covid-19 cases were diagnosed in the united states. only data from the united states were included in the present analysis to mitigate potential confounding effects (eg, variability of twitter use or google searches internationally) and inappropriate exclusions or inclusions of search terms borne from incorrect translation or nonaccounting of regional vernacular in this word-based study. data pertaining to daily covid-19 case numbers in the united states were collected from the new york times administered repository of covid-19-related case data. 7 data concerning covid-19 incidence and google search and tweet frequencies were examined individually via histograms and then in conjunction via scatterplots, revealing nonnormal distributions and nonlinear correlations. as such, spearman rank correlation coefficients were obtained to assess the relationship between google search and tweet frequency, as well as daily incidence of covid-19 in the united states. fisher r-to-z transformations were performed to compare spearman correlations. a p value of less than .05 was considered the threshold for significance. last, to understand tweet and google search trends in relation to mass media communications, we characterized ''peaks'' in tweet and google search frequency as being at least 2 standard deviations above their mean over the study period. correlations between covid-19 incidence and google search frequency pertaining to smell loss and nonsmell covid-19 symptoms from a similar time period (january 1, 2019, through april 8, 2019) in 2019 were obtained to confirm that infodemiological trends in covid-19 symptoms were indeed unique to the covid-19 era. tweet frequency concerning smell loss (0.539) was not as well correlated with daily covid-19 incidence as tweet frequency concerning covid-19 (0.848), nonsmell symptoms (0.761), and both and smell and nonsmell symptoms together (0.765) (tables 1 and 2). a significant peak in tweets concerning smell loss (.3 standard deviations greater than the mean for smell tweets) was seen around a widely read new york times article reporting a link between anosmia and covid-19 infection (march 22, 2020) 1 (see suppl. table sa in the online version of the article); data pertaining to march 22, 2020, and the 2 following days were excluded in 1 iteration of the analysis to help evaluate quantitatively the effect of discrete, lay media transmissions on twitter and google search trend correlations with covid-19 incidence. while the spearman correlation pertaining smell tweets (0.498) decreased, it remained statistically significant; more incremental decreases were seen in the correlation coefficients concerning nonsmell symptom (0.756). the change in the anosmia tweet correlation was not significant, however (p = .349). moreover, using twitter, we had the unique ability to identify tweets containing urls and retweets. the total number of tweets with urls and retweets included was 42,924, compared to 1444 when they were excluded, representing a 97% difference in tweet frequency (see suppl. table sa in the online version of the article). in addition, when excluding march 22 to 24, 2020, the correlation coefficient linking smell loss tweets (when including urls, retweets, and replies) improved significantly from 0.240 to 0.553 (p = .004) ( table 1 and figure 1 ). of twitter users who posted about smell loss, 52% were reported to be female, compared to 47% of all users posting about covid-19 (p \ .001). however, reported age distributions among users tweeting about smell loss (80% were 35 or older) were similar to the reported age distributions among users tweeting about covid-19 (78% were 35 or older). spearman correlations between google search queries concerning smell loss, covid-19, nonsmell symptoms, common anosmia therapies (''nasal irrigations,'' ''sinus rinse''), and daily covid-19 incidence were similarly assessed. the correlation between covid-19 incidence and searches pertaining smell loss (0.524) was less robust than searches about nonsmell symptoms (0.744) and covid-19 (0.893). there was no difference between dysgeusia (0.512) and smell loss correlations with covid-19 incidence. spearman correlations between daily covid-19 incidence and each individual nonsmell symptom were also assessed. both fever (0.749) and shortness of breath (0.732) were better correlated with covid-19 incidence than anosmia ( table 2) . no correlation was seen between fatigue search figure 2 ). no significant differences between tweet and google search correlations corresponding to smell loss (p = .401), all nonsmell symptoms (p = .392), covid-19 (p = .064), and covid-19 incidence were observed ( table 2 ). the covid-19 pandemic has affected millions of people in over 183 countries or territories, with wide-ranging sociopolitical and economic consequences. 8 while investigations into treatment methods are ongoing, there have been concomitant efforts to understand disease manifestations and patterns of spread using internet and social media platforms. the recognition of smell loss as a potential heralding and discriminant symptom of sars-cov-2 infection is impactful in both spreading awareness and as a potential temporally sensitive infodemiological tool to inform disease trajectory research. 9 in a timely publication by walker et al, 3 a significant correlation was identified between anosmia search volume (0.636) and covid-19 incidence in the united states between january 1 and march 25, 2020. the correlation reported in our study (0.564), while significant, was less strong, which may reflect a slight difference in study period. importantly, we found that google searches and tweets concerning covid-19, shortness of breath, fever, and all nonsmell symptoms combined were more strongly correlated with covid-19 incidence, which suggests that nonsmell loss-related parameters may be more sensitive to covid-19 incidence than anosmia ( table 2 ). this could be related to the fact that despite it being a distinctive symptom, the reported frequency of smell loss associated with covid-19 across the extant literature is variable but may be much lower than the other symptoms. 10, 11 moreover, as smell loss has been reported to be associated with a milder disease course, patients with a smell loss phenotype may be less likely to be tested for covid-19 and thus underrepresented in overall incidence. 12 therefore, it is plausible that anosmia may indeed be a sensitive infodemiological parameter for covid-19 incidence in the setting of more widespread testing. we also investigated dysgeusia separately, as it may represent more significant smell loss. however, there was no significant difference between dysgeusia (0.512) and smell loss correlations with covid-19 incidence. we also postulated that twitter might offer additional or unique insight into user health status as twitter represents a more expressive medium, allowing users to their post about their opinions, concerns, and symptoms. 13 however, correlations derived from tweets pertaining to smell dysfunction, nonsmell symptoms, and covid-19 were similarly sensitive to covid-19 incidence to their corresponding google searches ( table 2 ). more granular analysis may have allowed us to narrow our search to tweets explicitly relating to user reports of smell loss and improved the sensitivity of our correlations, but the natural language processing required was beyond the scope of the present study. we did, however, have the unique ability to gather user-specific demographic information with twitter. interestingly, a greater proportion of users tweeting about smell loss were female (52%), although a lesser proportion of users tweeting about covid-19 were female (47%). while this may simply reflect sexbased differences in care-seeking behavior, which has been demonstrated across a broad range of conditions, this trend might also offer putative insight regarding sex-oriented discrepancies in covid-19 presentation. 14 a significant spike in anosmia-related searches and tweets was seen after march 22, 2020, corresponding precisely to the publication of a widely read new york times article linking anosmia and sars-cov-2 infection (see suppl. table sa in the online version of the article). to quantify the effect of mass media communications on the correlation between smell loss tweets, searches, and covid-19 incidence, we excluded march 22 to march 24 to obtain new spearman correlation coefficients. only small, nonsignificant reductions in correlation were observed after excluding these 3 days. moreover, we aimed to mitigate the influence of mass media communications by excluding tweets containing urls and retweets, which we hypothesized were more likely to reflect user responses to media than their personal experiences. in analyzing tweets about smell loss that contained urls and retweets, however, we identified a significant improvement in the spearman coefficient linking anosmia tweets and covid-19 incidence (0.240 to 0.553) ( table 1) . this illustrates the significant potential influence in infodemiological data that can be introduced by mass media communication. as such, researchers must demonstrate an awareness of such influence when interpreting user-generated data in the context of understanding covid-19 disease trends. herein, we present a preliminary analysis illustrating the potential use of both twitter and google trends user data as potential corollaries for covid-19 incidence. while we identified several interesting relationships, we also highlight some pitfalls of infodemiological investigations in this rapidly evolving media-sensitive setting. first, we found that google search and tweet frequency concerning covid-19, and not anosmia or any other covid-19 symptom described in other infodemiological investigations, had the strongest correlation with daily disease incidence in the united states. moreover, we clearly found that mass media communication played a significant role in driving user behavior in both mediums (twitter and google search). this is a potential confounder of user-generated data that must be carefully accounted for in similar infodemiological inquiries. in addition, the background seasonal variation in other viral illnesses with similar symptomatology queried here, including influenza, could have affected the results. we suspect, however, that this influence is small given the overwhelming incidence of covid-19 relative to other viral illnesses. we hope that these preliminary findings and lessons learned may be levied to inform and enhance future covid-19-related studies using infodemiological methods. to that end, we recognize the possibility of future changes in findings borne from a more developed understanding of sars-cov2 prevalence (accounting for the rate of asymptomatic carriership, for example). google search and tweet frequency regarding more common covid-19 symptomatology (ie, fever and shortness of breath) are more robust indicators of daily disease incidence than anosmia. although twitter represents an alternative platform for infodemiological investigations, tweet frequency pertaining to covid-19-related symptoms was similar in sensitivity to google search trends. last, mass media communications represent important confounders that must be considered when correlating infodemiological trends with covid-19 incidence. bharat a. panuganti, substantial contributions to conception, acquisition, and design of study and analysis; composition of manuscript; final approval of work and full agreement with all components of the submission herein; aria jafari, substantial contributions to conception, acquisition, and design of study and analysis; composition of manuscript; final approval of work and full agreement with all components of the submission herein; bridget macdonald, substantial contributions to conception, acquisition, and design of study and analysis; composition of manuscript; final approval of work and full agreement with all components of the submission herein; adam s. deconde, substantial contributions to conception, acquisition, and design of study and analysis; composition of manuscript; final approval of work and full agreement with all components of the submission herein. lost sense of smell may be peculiar clue to coronavirus infection tracheotomy recommendations during the covid-19 pandemic the use of google trends to investigate the loss of smell related searches during covid-19 outbreak detecting influenza epidemics using search engine query data the parable of google flu: traps in big data analysis demographics section: age methodology calculation covid-19 data johns hopkins coronavirus resource center. covid-19 map google searches can help us find emerging covid-19 outbreaks association of chemosensory dysfunction and covid-19 in patients presenting with influenza-like symptoms anosmia and ageusia: common findings in covid-19 patients: otolaryngological manifestations in covid-19 self-reported olfactory loss associates with outpatient clinical course in covid-19 a model for mining public health topics from twitter. human language technology center of excellence epidemiology of anosmia in south korea: a nationwide population-based study competing interests: none. funding source: none. additional supporting information is available in the online version of the article. key: cord-265178-q7x7ec24 authors: lyócsa, štefan; baumohl, eduard; výrost, tomáš; molnár, peter title: fear of the coronavirus and the stock markets date: 2020-08-26 journal: financ res lett doi: 10.1016/j.frl.2020.101735 sha: doc_id: 265178 cord_uid: q7x7ec24 since the outbreak of the covid-19 pandemic, stock markets around the world have experienced unprecedented declines amid high uncertainty. in this paper, we use google search volume activity as a gauge of panic and fear. the chosen search terms are specific to the coronavirus crisis and correspond to phrases related to nonpharmaceutical intervention policies to fight physical contagion. we show that during this period, fear of the coronavirus – manifested as excess search volume – represents a timely and valuable data source for forecasting stock price variation around the world. the outbreak of the coronavirus (also referred to as covid-19) has heavily impacted society (dowd et al., 2020) and decimated the economy. stock markets around the world have witnessed unprecedented declines. on march 23, 2020, the u.s. benchmark stock market index s&p 500 lost as much as 35% of its value relative to its recent historical maximum achieved on february 19, 2020. in 5 historic fashion, within days, the magnitude of this decline became comparable to the financial crisis of october 2008, black monday in 1987, and the start of the great depression in october-november 1929. such evaporation of wealth has costly social and economic consequences, such as decreased consumption and even the reassessment of individual retirement plans (helppie mcfall, 2011) . research about the impact of the coronavirus pandemic on financial markets has naturally fol-10 lowed. okorie and lin (2020) find that financial contagion occurs during the coronavirus crisis, and akhtaruzzaman et al. (2020) also highlights that financial firms contributed to the contagion more than nonfinancial firms. the results of baumhl et al. (2020) indicate that the systemic risk among banks around the world and the density of the spillover network have never been as high -not even during the 2008 financial crisis -as they have been during the covid-19 pandemic. corbet et al. (2020a) document that the coronavirus pandemic particularly negatively affected companies with names related to coronavirus, even though these companies were unrelated to the virus. during the covid-19 crisis, gold acted as a safe haven , while results for bitcoin are less conclusive: goodell and goutte (2020) suggest that bitcoin acted as safe haven, while , and corbet et al. (2020b) conclude the opposite. ashraf (2020) find that stock markets 20 responded negatively to the growth in confirmed cases of covid-19. further topics for research are suggested in goodell (2020) . with a sample of the largest 10 stock markets (united states (us), united kingdom (uk), japan (jp), france (fr), india (in), canada (ca), germany (de), switzerland (ch), south korea (kr) and australia (au)), covering approximately 80% of global market capitalization, we show in this 25 paper that during the 'orona crash', stock markets around the world reacted to fear of the coronavirus. to measure fear, we rely on internet searches of corona-related terms. recently, bento et al. (2020) showed that the response of the general public to news about local covid-19 cases is to search for more information on the internet. internet searches have been shown to be useful in many applications, e.g., tracking influenza-like epidemics in a population (ginsberg et al., 2009) . the idea of using sentiment or fear to explain stock market volatility is certainly not new; several recent studies have used news, vix, twitter posts and other proxies to measure investors' sentiment and fear about the future (e.g., whaley, 2000; zhang et al., 2011; huerta et al., 2011; smales, 2014 smales, , 2017 . however, our study is the first to address the predictive power of google searches on stock market volatility during the covid-19 pandemic. our results show that high google search volumes 35 for covid-19 predict high stock market volatility in all markets in our sample. the conclusion that covid-19 increases stock market volatility accords with sharif et al. (2020) , zaremba et al. (2020) and zhang et al. (2020) . however, our work complements theirs, as sharif et al. (2020) the rest of the paper is organized as follows. section 2 presents the data and describes the construction of the variables. section 3 presents the methods and results. section 4 concludes. data on price variation are retrieved from the oxford-man institute's realized library 1 . we use data from the following ten indices: the s&p 500 (us), ftse 100 (uk), nikkei 225 (jp), cac 40 (fr), nifty 50 (in), s&p/tsx composite (ca), dax (de), smi (ch), kospi (kr), and all ordinaries (au). the google trends are retrieved using a program package in r (massicotte and eddelbuettel, 2020) . data are available upon request. thus, to capture fear of the coronavirus, we use only data from google trends, i.e., a search volume index (sv i t,j ), where index j denotes a specific search term. the idea proposed by preis et al. (2013) is that prior to trading, investors search for information; therefore, such data lead future trends, particularly declines in the financial market. we retrieve daily individual search volume indices that are normalized to the range from 0 to 100 for the following 19 english words: corona', world health 55 organization', virus', sars', mers', epidemic', pandemic', symptom', infected', spread', outbreak', social distancing', restriction', quarantine', suspend', travel', lockdown' , and postpone'. these terms are related to the coronavirus crisis and are thus unlikely to have been predictive of market uncertainty in the past. we aggregate search intensity across these terms by taking the average across all individual indices for each day t. the first principal component was also highly correlated with the 60 average we used; therefore, we opted for the simpler average. the result is the average search volume index, asv i t . as an alternative to google searches, we considered using data on nonpharmaceutical interventions (npis) implemented by governments around the world. we specifically considered data on interventions in these four categories: social distancing, movement restrictions, public health measures, and 65 social/economic measures. (all of the categories included several pandemic-related policy responses 2 ). 1 http://realized.oxford-man.ox.ac.uk/data/download 2 for example, (1) social distancing includes schools closures, public services closures, lockdowns, and limits on the challenge posed by npis is that not only does the public tend to be informed about such measures in advance, but also such measures are publicly discussed before they are agreed upon. consequently, npis cannot be properly synchronized with market data. using google searches is free of such issues. to capture the attention of the public to npis and the spread of the coronavirus, we use search terms 70 that are derived from the names of various npis. to study how changing patterns in search activity are related to market uncertainty, we follow the work of da et al. (2011) and calculate the abnormal search volume activity: the asv i t is generally considered as a measure of attention (da et al., 2011) , and attention can 75 have various causes. in the case of covid-19, it could be fear, curiosity, or search for some practical information, e.g., how to create a face mask. our interpretation is that at the outbreak of covid-19, given that the speed, extent and the negative consequences of the pandemic on society were largely public gatherings; (2) movement restrictions include visa restrictions, travel restrictions, international flight suspensions, border closures, domestic travel restrictions, border checks, and additional health/document requirements upon arrival; (3) public health measures include health screenings in airports and border crossings, introducing quarantine policies, awareness campaigns, and strengthening the public health system; (4) social/economic measures include health screenings in airports and border crossings, introducing quarantine policies, awareness campaigns, and strengthening the public health system. unanticipated, the sudden increase in the interest, as captured by asv a t , primarily represents fear of this pandemic. on the other hand, in later stages of covid-19, once people were familiar with it, 80 people searched for information for various other reasons, and therefore, in general, asv a t represents attention rather than fear. as the speed, extent and the consequences of the coronavirus crisis were largely unanticipated, we interpret the asv a t as a gauge of panic and fear. the higher the value of asv a t is, the larger the increase in the interest of the population in coronavirus-related events, fear and panic; consequently, 85 we hypothesize larger market uncertainty. the same underlying idea has been used by zhang et al. (2011) , who argued that emotional outbursts of any kind posted on twitter can give a prediction of how the stock market will do the following day. panel b of table 1 shows the key statistics of asv a t across 10 developed markets. the shocks in search activity show signs of short-term persistence that is much smaller than the persistence of market uncertainty (see panel a of table 1 ). to measure market uncertainty, we resort to the daily variance of market returns (realized variance) calculated from high-frequency data. the higher the realized variance is, the higher the market uncertainty. specifically, we model the annualized daily variance as follows: is the i th intraday continuous return and p t,i is the value of a stock market index on day t at intraday time i = 1, 2, ..., m . the term j t = 100 × (lnp t,1 − lnp t−1,m ) 95 is the return between the closing value of the index on day t − 1 and the opening value on day t. the overnight price variation is added because the closing and opening values of the market index often differ. for the intraday component, we use the common 5-minute calendar sampling scheme to sample index values p t,i . a standard assumption for the data generating process of p t is d log(p t ) = µdt+σ t dw t , where µ t is the drift parameter, σ t is the instantaneous volatility, and w t is standard brownian motion. the integrated variance over a time span [t-∆t,t], iv t = t t−∆t σ 2 s ds, is not directly observable, but andersen et al. (2001) show that the integrated variance can be approximated from the sum of the squared intraday returns, which are observable from past intraday returns. we visualize market development and search intensity for the largest market index, the s&p 500. the upper panel of figure 1 shows the value (p t , left y-axis) and realized variance (rv t , right y-axis) of the s&p 500 index over our sample period from december 2, 2019 to april 30, 2020. we observe that during the onset of the corona crash, the value of the market declined, while uncertainty in the market increased to extreme levels. figure 1 also shows the average daily realized variance over 20 years prior to our sample window; the average daily realized variance reached a modest value of 278.53 110 (red-line). the average over our sample for the u.s. market is much higher at 1776.98. the lower panel in figure 1 plots the average search intensity in the u.s. (asv i u s t ) over time and the corresponding abnormal search volume activity (asv a u s t ). figure 1 shows that the period of extreme market uncertainty coincides with a period of higher attention of investors to corona events. during trading days from march 12−16, 2020, when market variance reached its highest values, sv i t,j the rv w t is the weekly volatility component, calculated as 4 −1 t−4 s=t−1 rv s , i.e., the daily and weekly components do not overlap. the original har model of corsi (2009) also includes a monthly volatility component, but our conclusions are not influenced if the monthly component (which is not significant) is included. we use the simplified version, as we are also using a shorter sample period. 135 we use the log-log specification to address the positive skew of the variances, while the estimated β 1 and β 2 coefficients can be interpreted as the % change in the rv t+1 given a 1% change in rv t , i.e., the elasticity. panel a of table 1 shows statistics of the log of the realized variance across 10 markets. the autocorrelation of the volatility at the 10 th lag is still considerable: thus, unconditional volatility is highly persistent, even during our sample period of the corona market crash. the results from the benchmark model are reported in table 2 . panel a reveals that the behavior of the variance is very similar across countries -variance is highly persistent, and the variance from the previous day and week provides considerable information about the variance on the subsequent day. a 1% increase in variance on the previous day accounts for at least a 0.298% (india) or even up to a 0.588% (us) increase in realized variance on the next day. for several markets, weekly components 145 are even stronger drivers of market uncertainty. additionally, the benchmark models already appear to be reasonably well specified; i.e., no serial dependence (see the empirical likelihood (el) test) with almost always homoscedastic (see white's test) residuals. second, we add local abnormal search volume activity: the results reported in panel b (table 2) show that the abnormal search volume activity improves 150 the predictability of market uncertainty on the subsequent day. the asv a t is positive for all markets and significant for all markets except south korea, thus suggesting that when search activity related to corona information increased, price variation in stock markets increased the following day. when abnormal search volume activity increases 2 standard deviations (sds) (see panel b in table 1 ) above the average, the market's realized variance effect almost doubles (rv t , particularly for the u.s., japan, india and germany). third, we add the global abnormal search volume activity: the results reported in panel c (table 2) show a similar picture with even larger coefficients for several markets, thereby suggesting not only that the fear is global but also that the economies are highly interconnected. therefore, these results confirm our previous observation that local and global 160 searches for coronavirus are very similar and, therefore, have the potential to affect markets similarly. in south korea, global google searches work much better than local google searches. the likely reason is that google is not the most popular search engine in south korea. ashraf (2020) showed that the negative market reaction was strong during the early stages of the 165 covid-19 pandemic. stock markets around the world quickly responded to the pandemic, but this response may vary over time depending on the stage of the outbreak. furthermore, bento et al. (2020) shows that most interest in covid-19 was found during the first weeks after the first positive cases were confirmed. if our results are driven by the period from february and march, i.e., during the initial onset of 170 the covid-19 pandemic, it strengthens our interpretation that fear drove market movements. we therefore estimated our regressions over the period from april 1, 2020, to july, 31, 2020. 3 during the post-fear period, asv a t reaches less extreme values. given that the general public already knew about the covid-19 at that time, this finding is expected. consequently, for the april to july period, the results are also very different. regression coefficients loaded at asv a t are not 175 positive and significant as before, but are often insignificant and, in some cases, are even negative. the negative coefficient suggests that as the covid-19 pandemic progressed and the general public increased its awareness about covid-19 and/or pandemic-related policy responses, market volatility decreased. the fit of these models also decreased dramatically. it therefore appears that for this period, the abnormal interest of the general population in covid-19 is not a market-moving factor. 180 we therefore interpret our results from the december to may period as being a manifestation of fear, not of mere attention or curiosity. during the outbreak of the covid-19 pandemic, markets around the world lost an extreme amount value in a short period, strongly negatively affecting societies. we show that at least part of this 185 turbulence was driven by short-term investors' sentiment -that is, fear created by the coronavirus. the level of this fear is measured by google searches, and this fear has a significant predictive power for future stock market uncertainty. the observation that google searches for coronavirus are correlated with price variation is perhaps unsurprising. the research linking stock markets' movements to investors' attention and sentiment 190 has become quite extensive over the last few years (e.g., hamid and heiden, 2015; bijl et al., 2016; dimpfl and jank, 2016; kim et al., 2019; audrino et al., 2020) . however, our results show that google searches for coronavirus are not simply correlated: these searches predict variance in the future for every country we considered. this result can be utilized in risk management models. during uncertain, unprecedented periods, google searches present a valuable data source that might improve assessment 195 of market risks. the term 'coronavirus' will probably be the most-searched term in the history of google trends. notes: the superscripts a, b, and c denote statistical significance at the 10%, 5%, and 1% levels, using a random block length bootstrapping scheme with 1000 replications, as in patton et al. (2009) . the el's test is the p-value of the test of no serial correlation in the residuals of escanciano and lobato (2009) , and white's test is a nonparametric unweighted bootstrap test of no heteroscedasticity in residuals (cribari-neto and zarkos, 1999) . financial contagion during covid-19 crisis the distribution of realized stock return volatility stock markets reaction to covid-19: cases or fatalities? the impact of sentiment and attention measures on stock 205 market volatility from physical to financial contagion: the covid-19 pandemic and increasing systemic risk among banks evidence from inter-210 net search data shows information-seeking responses to news of local covid-19 cases google searches and stock returns are cryptocurrencies a safe haven for equity markets an international perspective from the covid-19 pandemic safe haven or risky hazard? bitcoin during the covid-19 bear market aye corona! the contagion effects of being 220 named corona during the covid-19 pandemic the contagion effects of the covid-19 pandemic: evidence from gold and cryptocurrencies a simple approximate long-memory model of realized volatility bootstrap methods for heteroskedastic regression models: evidence on estimation and testing in search of attention can internet search queries help to predict stock market volatility? european demographic science aids in understanding the spread and fatality rates of covid-19 an automatic portmanteau test for serial correlation detecting influenza epidemics using search engine query data covid-19 and finance: agendas for future research co-movement of covid-19 and bitcoin: evidence from wavelet 240 coherence analysis forecasting volatility with empirical similarity and google trends crash and wait? the impact of the great recession on the retirement plans of older americans the impact of tarp bailouts on stock market volatility and investor fear searching for safe-haven assets during the covid-19 pandemic google searches and stock market activity: 250 evidence from norway 2020. gtrendsr: perform and display google trends queries stock markets and the covid-19 fractal contagion effects correction to automatic block-length selection for the dependent bootstrap by d. politis and h. white quantifying trading behavior in financial markets using google trends covid-19 pandemic, oil prices, stock market, geopolitical risk 260 and policy uncertainty nexus in the us economy: fresh evidence from the wavelet-based approach news sentiment and the investor fear gauge the importance of fear: investor sentiment and stock market returns the investor fear gauge infected markets: novel coronavirus, government interventions, and stock return volatility around the globe financial markets under the global pandemic of covid-19. finance 270 research letters predicting stock market indicators through twitter i hope it is not as bad as i fear writing -original draft preparation, writing -review & editing, visualization, project administration key: cord-339642-3trpona9 authors: obeidat, rand; alsmadi, izzat; bani bakr, qanita; obeidat, laith title: can users search trends predict people scares or disease breakout? an examination of infectious skin diseases in the united states date: 2020-06-08 journal: infect dis (auckl) doi: 10.1177/1178633720928356 sha: doc_id: 339642 cord_uid: 3trpona9 background: in health and medicine, people heavily use the internet to search for information about symptoms, diseases, and treatments. as such, the internet information can simulate expert medical doctors, pharmacists, and other health care providers. aim: this article aims to evaluate a dataset of search terms to determine whether search queries and terms can be used to reliably predict skin disease breakouts. furthermore, the authors propose and evaluate a model to decide when to declare a particular month as epidemic at the us national level. methods: a model was designed to distinguish a breakout in skin diseases based on the number of monthly discovered cases. to apply this model, the authors correlated google trends of popular search terms with monthly reported rubella and measles cases from centers for disease control and prevention (cdc). regressions and decision trees were used to determine the impact of different terms to trigger the occurrence of epidemic classes. results: results showed that the volume of search keywords for rubella and measles rises when the volume of those reported diseases rises. results also implied that the overall process was successful and should be repeated with other diseases. such process can trigger different actions or activities to be taken when a certain month is declared as “epidemic.” furthermore, this research has shown great interest for vaccination against measles and rubella. conclusions: the findings suggest that the search queries and keyword trends can be truly reliable to be used for the prediction of disease outbreaks and some other related knowledge extraction applications. also search-term surveillance can provide an additional tool for infectious disease surveillance. future research needs to re-apply the model used in this article, and researchers need to question whether characterizing the epidemiology of coronavirus disease 2019 (covid-19) pandemic waves in united states can be done through search queries and keyword trends. infectious skin diseases encompass a vast array of conditions that range in severity from mild to life-threatening. the clinical presentation of infectious skin diseases varies based on the type of pathogen involved, the skin layers and structures affected, and the underlying medical condition of the patient. infectious skin diseases represent common diagnoses made by dermatologists, by primary care physicians, and in the emergency room. 1 rubella, in particular, though a mild, vaccinepreventable skin disease, is of high public health importance owing to the teratogenic effects that can result from congenital rubella infection (cri), leading to miscarriage, fetal death, or birth of an infant with congenital rubella syndrome (crs). 2, 3 rapidly identifying an infectious disease outbreak is critical, both for effective initiation of public health intervention measures and timely alerting of government agencies and the general public. 4 a vast amount of real-time information about infectious disease outbreaks can be found in various forms of web-based data streams. studies show that health care providers rely on online search results in obtaining more information about diseases, symptoms, drugs, and other related information. also, the research showed that doctors find searching online very helpful to get information about tracking geographical locations of disease. google search queries are the most commonly used data source for search studies around the world. for example, google's search engine has been used to detect influenza epidemics in areas with a large population of web search users because of its high correlation with the percentage of physician visits if a patient has influenza-like symptoms. 5 research studies sought to study the association between disease outbreak and online search keywords and terms. for example, a study by polgreenet al 6 examined the relationship between searches for influenza and actual influenza occurrence. another study by yom-tov and fernandez-luque 7 collected data from a major internet search engine, while people seek information about the measles, mumps, and rubella (mmr) vaccine. the authors focused on developing an automated way to score internet search queries and web pages to examine how people use internet search engines to garner information on vaccination. recognizing the need for up-to-date data to inform researchers, policymakers, public stakeholders, and health care providers if search queries can be used to reliably predict skin disease breakouts, we correlated google trends popular search terms with monthly reported rubella and measles cases from 2004 to 2018. so, this study provides analysis and evaluation for the association between monthly reported rubella and measles cases and google trends popular search terms that can be used to predict a future outbreak of infectious skin disease case. several research studies used google trends to answer research questions within health care domains. 8 some of these studies examined and confirmed the correlation between disease outbreaks and online search keywords and trends. 9 in 2009, ginsberg et al 5 stated that the google flu trends predictions were 97% accurate compared with the centers for disease control and prevention (cdc) data. cdc was also testing google flu trends in the united states, and the preliminary finding suggests that google flu trends can detect regional outbreaks of influenza 7 to 10 days earlier than conventional cdc surveillance. 9, 10 the correlation between google trends and diseases surveillance was also assessed in several countries such as india, 11 south korea, 12 south china, 13 and spain. 14 however, those studies did not propose any unique search terms that can correlate with diseases predictions. google trends were deployed to detect/estimate many disease outbreaks such as influenza, 15, 16 dengue, 17, 18 ebola, 19 and lyme. 20 few studies analyzed skin-related diseases using google trends. bloom et al 21 extracted data from google trends to evaluate whether population inquisitiveness on melanoma and skin cancer was correlated with a lower incidence. they found that the general populations' interest in learning about skin cancer increases during the summer month. hopkins and secrest 22 used google trends data queried using several search terms (sunscreen, sunburn, skin cancer, and melanoma) in the united states. then, time-matched search term data were correlated with melanoma outcomes data from surveillance epidemiology and end results program and united states cancer statistics. in another study, hopkins and secrest 22 explored international trends in english-speaking countries including (united states, united kingdom, canada, australia, and new zealand) several search terms that are used to better guide skin cancer prevention campaigns. hopkins and secrest 23 assessed the correlations between search terms, time, and melanoma outcomes for each country. none of the previous studies correlated google trends popular search terms with certain infectious skin diseases including rubella and measles reported from cdc. in this work, google trends was used to propose unique search terms that can correlate with epidemic disease prediction. for this purpose, we collected data and we used machine learning methods to evaluate a dataset of search terms to determine if search queries and terms can be used to reliably predict skin disease breakouts. in this study, it is important to use different classifiers to have more confidence in the results and compare those different classifiers based on accuracy. therefore, we have created a new supervised classification model that uses a support-vector machine (svm) model, linear regression (lr), and decision tree (dt) to evaluate each disease breakout prediction. in the first part of this section, we provide an overview of the model we used. then, we explain the data used in this study. we finish this section by presenting the proposed model for disease epidemic classification and algorithm. using svm is a supervised machine learning technique that is widely used in classification and regression problems. the main objective of svm algorithm is to find a hyperplane in an n-dimensional space that distinctly classifies the data points, where n is the number of features. to separate 2 classes of data points, infinite number of hyperplanes could be found. in the svm, the main objective is to find a plane that has the maximum margin. a separating hyperplane can be written as the following equation: . ., wn} is a weight vector and b a scalar (bias). for example, in two-dimensional (2d) it can be written as: the hyperplane defining the sides of the margin is as follows: h1: 0 + 1 1 + 2 2 1 for = +1, any training tuples that fall on hyperplanes h1 or h2 are support vectors. we used svm as a classifier where in the applied svm model, rows represent months and columns represent relevant google trends keywords that were extracted from several obeidat et al 3 cycles. dates used in svm were matched for google trends keywords and cdc reported diseases. table 1 shows the last 2 columns of the svm matrix that we created from the number of reported cases, one of the columns as a continuous value and the last column as a binary target, based on our epidemic formula. in the lr, given n observations, where an output vector y with dimension n × 1 and p inputs x1, x2, . . ., xp, where each input vector being of dimension n × 1, lr assumes that the regression function e(y|x) is linear in the inputs. y is computed based on the following equation: where ε is the error term. lr model is applied to predict diseases, such as measles, using many search terms (in terms of their significance and estimated values). decision tree is a supervised learning algorithm which has a flowchart-like tree structure. in the dt, each internal node (non-leaf node) represents a test on an attribute. each branch is an outcome of the tested condition of on an attribute, and each leaf node or terminal node holds a class label. decision tree classification model was employed to study the different terms impact on diseases predication. centers for disease control and prevention data. data were collected on reported cases for each disease in the united states over the period from 2004 to 2018. reported cases of diseases by months were collected and maintained from public cdc publications. then, the authors did some data transformation to aggregate data across all united states to monthly basis to match google trends data. google trends data. initial sets of relevant keywords were created to each disease and used them to extract google trends data. specifically, in this article, the authors produced a dataset of popular search terms for 2 infectious skin diseases such as rubella and measles that can be used to predict future skin disease breakouts. google reports search terms on a monthly basis were accumulated. the proposed model for disease epidemic classification and algorithm. the authors designed an "epidemic" model to distinguish a breakout of diseases based on the number of monthly discovered cases, as well as to decide whether a certain month counts as "epidemic." the authors observed final values collected from cdc and google trends and then decided to make the cut-off in epidemic class based on the reported values for each disease. epidemic class value will be 1, or else will be zero (ie, yes, 1 or no, 0). table 1 shows that the level of increase in a month from previous month was calculated using this formula: (current month-previous month)/previous month. transient month increase or decrease was eliminated. as a result, this increase will be considered epidemic if it occurs in 3 consecutive months. the authors made this algorithm as an approximation of how to flag a month as an epidemic month. this model was applied and validated to distinguish a breakout in rubella and measles skin diseases based on the number of monthly discovered cases. the process of the creation of rubella popular search terms has the following main steps: • • start by evaluating the single term (for each skin disease) in google trends across the available period (from 2004 to 2018), with a total of 168 records. the numbers given for the popular terms are in the form of percentages (ie, from 0 to 100), rather than actual volumes of search terms. • • in the second step of keyword selection, we use google trends to expand our search terms. starting step 1, we analyze the "related queries" section in google trends and we extract the "top" and "rising" search terms as shown in table 2 . • • as shown in table 2 , google trends distinguishes terms breakout and rising keywords in terms of how quick and long such terms have been on the search rise. for a search term to be selected in the collected dataset, it should have the following inclusion criteria: (1) search terms are extracted only from the "related queries" section in google trends results. it should be listed in the "top" or listed with more than 90% "rising" terms. (2) it should be repeated for more than 4 times in the "related queries" from different results (ie, from different initial search terms). this was necessary to eliminate out term that are "outliers" (if they show up in only one related term). an svm model was used to evaluate each disease breakout prediction based on collected features in the different experiments. correlations (pearson and spearman) were used between google trends of popular search terms and monthly reported rubella and measles cases from cdc. in addition, regressions and dts were used to determine the impact of different terms to trigger the occurrence of epidemic classes. in our svm model (table 3 , rubella svm sample), rows represent months and columns represent relevant google trends keywords that were extracted from several cycles. dates were matched for google trends keywords and cdc reported diseases. the count column was retrieved from google trends and represents the popularity of those relevant keywords in that particular month. records represent monthly data for both disease volume and google trends selected keywords. for rubella disease, the volume of reported cases was small. in addition, in our dataset, we did not find reports for many other months (ie, missing values). this impacted overall prediction accuracy. table 4 shows the results of lr prediction on rubella svm. we showed search terms with the lowest p values (ie, significant prediction results). however, their estimate values are low, which indicates a low overall impact on disease prediction. accuracy of prediction for measles lr model is better for many search terms (in terms of their significance and estimate values; table 5 ). one main reason for such better accuracy is the large number of reported cases for measles and also the fact that we have much fewer missing values for measles' case. for each one of the experiments to extract relevant keywords, we evaluated correlations (pearson and spearman) between popular terms of google trends and disease arrays. in terms of correlation, no significant positive or negative correlation is shown in the volume of those terms and cases volumes. however, the highest keywords for rubella in terms of correlation (negative or positive) were titer, rubeola, crs (positive), rubella pregnancy, and rubella rash. decision tree classification model was employed as the model has a categorical target class to study the different terms impact. with more than 95% accuracy, figure 1 shows overall accuracy metrics. figure 1 shows a high true positive (tp) rate and a very low false positive (fp) rate which implies very acceptable accuracy in all recorded performance metrics (ie, precision, recall, mcc, receiver operating characteristic [roc] area, and precision-recall curve [prc] area). due to size limitation, we show a summary snapshot from measles dt in figure 2 . this figure summarizes search terms with a significant impact on our proposed epidemic class. this figure shows also minimum weight for the search term to trigger the occurrence of the epidemic class. in other words, if people are searching for more than this percentage on this particular term, then the rise in this disease is significant. the dt shows the keywords that decide the target class (whether a month is an epidemic or not), their cut-off value to switch the target class from yes (epidemic) to no, and also how many instances in the dataset in that category. this article aimed to evaluate a dataset of search terms to determine whether search queries and terms can be used to reliably predict disease breakouts. a model was proposed and evaluated to decide when to declare a particular month as epidemic at the us national level. in this study, the authors applied the model on 2 infectious skin diseases such as rubella and measles. by using lr as a regression method, we showed that the search terms with the lowest p values estimate values that are low, which indicates a low overall impact on disease prediction. by using the lr, we also found that the accuracy level of prediction for measles is higher than the accuracy of prediction for rubella using several search terms as shown in tables 4 and 5 . in addition, the dt classification model was employed as the model for classification with more than 95% accuracy. the dt model successfully shows that the keywords features can be used to classify whether a month is an epidemic or not with accuracy reach to 95%. in this study, we found that people search for rubella and measles diseases throughout the year. results showed that the volume of search keywords for rubella and measles rises when the volume of reported diseases rises. due to the small volume of reported cases for rubella, it is found that the accuracy level of prediction for measles is higher than the accuracy of prediction for rubella. despite some challenges related to missing values in certain months, the results implied that the overall process was successful and should be repeated with other diseases. such a process can trigger different actions or activities to be taken when a certain month is declared as "epidemic." one interesting observation is that the query volumes considerably vary according to the searched term. however, this research has shown great interest in vaccination against measles and rubella. this study has some limitations. at first, we were weighing our options to use us data at the national level or state by state. however, based on data availability, we reported analysis only at us national level. in the future, and based on data availability in the cdc, we will analyze historical data on several years per state. for google trends, one major limitation we have to deal with in google trends is that google trends aggregates relative not absolute data. all data reported are relative (ie, in percentage from 0% to 100%) rather than actual volumes of search terms. in the era of online information overload, can users search trends predict diseases outbreak? to address this question, this study aimed at evaluating a dataset of search terms from 2004 to 2018, by developing and evaluating a model to decide when to declare a particular month as epidemic at the us national level. the findings suggest that the search queries and keyword trends can be truly reliable to be used for the prediction of disease outbreaks, and search-term surveillance can provide an additional tool for infectious disease surveillance. future research needs to re-apply the model used in this article, and researchers need to question whether characterizing the epidemiology of coronavirus disease 2019 (covid-19) pandemic waves in the united states can be done through search queries and keyword trends. infectious skin diseases: a review and needs assessment control of rubella and congenital rubella syndrome (crs) in developing countries, part 1: burden of disease from crs high burden of congenital infection and spread to canada early detection of disease outbreaks using the internet detecting influenza epidemics using search engine query data using internet searches for influenza surveillance information is in the eye of the beholder: seeking information on the mmr vaccine through an internet search engine the use of google trends in health care research: a systematic review google trends: a web-based tool for real-time surveillance of disease outbreaks influenza forecasting with google flu trends google search trends predicting disease outbreaks: an analysis from india correlation between national influenza surveillance data and google trends in south korea using google trends for influenza surveillance in south china diseases tracked by using google trends assessing google flu trends performance in the united states during the 2009 influenza virus a (h1n1) pandemic reassessing google flu trends data for detection of seasonal and pandemic influenza: a comparative epidemiological study at three geographic scales evaluation of internet-based dengue query data: google dengue trends using web search query data to monitor dengue epidemics: a new model for neglected tropical disease surveillance assessing ebola-related web search behaviour: insights and implications from an analytical study of google trendsbased query volumes the utility of "google trends" for epidemiological research: lyme disease as an example google search trends and skin cancer: evaluating the us population's interest in skin cancer and its association with melanoma outcomes public health implications of google searches for sunscreen, sunburn, skin cancer, and melanoma in the united states an international comparison of google searches for sunscreen, sunburn, skin cancer, and melanoma: current trends and public health implications data collection: ro rand obeidat https://orcid.org/0000-0001-8271-3829 key: cord-351108-wfik975q authors: cherry, george; rocke, john; chu, michael; liu, jacklyn; lechner, matt; lund, valerie j.; kumar, b. nirmal title: loss of smell and taste: a new marker of covid-19? tracking reduced sense of smell during the coronavirus pandemic using search trends date: 2020-07-16 journal: expert review of anti-infective therapy doi: 10.1080/14787210.2020.1792289 sha: doc_id: 351108 cord_uid: wfik975q objectives: it has been demonstrated that reduction in smell and/or taste is the most predictive symptom in sars-cov-2/covid-19 infection. we used google trends to analyze regional searches relating to loss of smell and taste across italy, spain, france, brazil, and the united states of america and determined the association with reported covid-19 cases. methods: in order to retrieve the data, we built a python software program that provides access to google trends data via an application program interface. daily covid-19 case data for subregions of the five countries selected were retrieved from respective national health authorities. we sought to assess the association between raw search interest data and covid-19 new daily cases per million for all regions individually. results: in total, we yielded 2188 sets of google trends data which included 548 time series of 4 anosmia and ageusia search concepts over the study period for 137 regions. these data indicated that differences in search interest for terms relating to anosmia and ageusia, between regions, is associated with geographical trends in new covid-19 cases. conclusions: we feel that google search trends relating to loss of smell can be utilized to identify potential covid-19 outbreaks on a national and regional basis. the loss or reduction of the sense of smell has been widely reported as a key symptom of sars-cov-2/covid-19 infection and, in a significant proportion of cases, it has been demonstrated to be the only complaint [1] [2] [3] . the positive predictive value for olfactory dysfunction (od), or loss of smell, for covid-19 positivity has been demonstrated to be over 60%; higher than any other associated symptoms [4] . the world health organization (who) integrated loss of smell and/or taste to their official list of associated symptoms and public health england have recently updated their recommendations for self-isolation to include olfactory dysfunction. self-isolation and social distancing are seen as key public health strategies in controlling the covid-19 outbreak [5] . internationally, governments are now looking to reduce restrictions on their populations to prevent economic damage. as prohibitive measures are reduced, it is likely that there will be further outbreaks of covid-19 [6] . moving forward, accurate tracking of covid-19 cases and early quarantining of infected people or populations will be vital in reducing transmission and preventing secondary outbreak [6] . effective strategies will need to be adopted by leaders and public health bodies internationally to reflect this. with a worldwide need for polymerase chain reaction (pcr) tests and a significant associated economic cost, particularly in low-to-middle income countries, it is paramount to develop alternative strategies to track outbreaks aside from pcr testing and tracing [6, 7] . walker et al. have demonstrated the utility of search term analysis, using google trends, in determining significant associations between reported covid-19 cases on a countrywide level [8] . using search interest data to infer population-wide behavior in developed countries has recently become possible. this is due to an increase in mobile internet usage from 17% to 78%, in the united kingdom between 2008 and 2018 and through the adoption of unified browser uniform resource locator (url) and search bars after 2010 [9] . the two major mobile platforms use google which drives almost all mobile search traffic through this search engine. this has reinforced a new behavior whereby people quickly and easily search for anything, wherever they are, almost exclusively with google. we used google trends to analyze regional searches relating to loss of smell and taste across italy, spain, france, brazil, and the united states of america (usa) and determined the association with reported covid-19 cases using a self-developed software programme (python). moving forward, regional and sub-regional tracking of outbreaks using this or a similar method could help a targeted local public health strategies and testing. this has the potential to reduce the demand on costly pcr tests and allow areas that are not affected by new infections to avoid restrictive measures and open up their local economies and healthcare system. we chose the five countries worst affected by covid-19 according to johns hopkins global covid-19 data; usa, uk, spain, italy, and france [10]. however, google trends subdivides the uk into the home nations only (england, scotland, wales, and northern ireland) making further regional analysis impossible. therefore, brazil was used in place of the uk. google trends is an open access platform providing data on the amount of search requests performed using google. for a chosen search term, date range and geographical region google trends data contains the relative changes in volume of search requests over time and between regions. of particular usefulness, the geographical boundaries use iso-3166 standard national subregions which typically match covid-19 reporting from the parent countries. google trends data is normalized over the time period and geography in question against all other searches that took place. a value of 100 indicates the highest popularity for that term and 0 the lowest popularity. similarly, for geographical comparisons, a score of 100 for a region indicates the highest relative popularity and 0 the lowest -for that country or region within a country [11] . we defined four concepts that individuals may perform a google search for when experiencing anosmia or ageusia: • loss of sense of smell • sense of smell • loss of sense of taste • sense of taste these four concepts were manually translated using the google trends related queries tool in order to find the appropriate semantic translation and grammar -as opposed to a literal translation. table 1 shows the results of this. national subregions were defined by their iso-3166 codeswith the exception of france as google trends uses the pre-2016 regional boundaries [12] . the time period of interest for google trends data started when each country began recording subregional covid-19 data and ended on 17 may 2020. the usa was an exception as the previous method exceeded google trends 90-day window for daily data aggregations, therefore the date any us state passed 10 cases of covid-19 was taken as the start date. search interest data were aggregated on a daily basis within the period of interest. in order to retrieve the data, we built a python software program, which includes the pytrends open source library that provides access to google trends data via an application program interface (api) [13] . using the pytrends request methods we collected search interest data over time for each translated term per subregion of each country. a second set of search interest data was collected for individual days within the time period per search term and country which provided relative values for search interest between subregions. daily covid-19 case data for subregions of the five countries selected were retrieved from respective national health authorities [14] [15] [16] [17] [18] . regional population data were supplied with covid-19 data from brazil and the usa. census population data were retrieved separately for italy, spain, and france [19] [20] [21] . french covid-19 and population data were reorganized from department level grouping into the pre-2016 regional structure in order to match the search interest geographies used by google trends. further data processing was performed using our selfdeveloped software program. unadjusted search interest data could be used for within region analyses over time. to conduct between region analyses, search interest over time per region was combined with national search interest data providing relative weightings between regions producing a figure for daily weighted search interest. covid-19 data were normalized against the population of each region producing a value for new cases per million. normalizing covid-19 case data were required to adjust for population differences in order to make between region comparisons between regions within a country. raw search interest data, weighted search interest data, and covid-19 new daily cases per million data were smoothed using a 7-day moving-mean to reduce the noise introduced by recording errors and low data volumes. we then sought to assess the association between smoothed raw search interest data and smoothed covid-19 new daily cases per million for all regions individually. following this we looked broadly at the number of regions with significant associations and the strength of any association. a second analysis pooled the smoothed, weighted search interest data for all regions within a country and the smoothed covid-19 new daily cases per million. this second analysis was an attempt to understand whether search interest data could provide insights into geographical as well as temporal changes in anosmia and ageusia secondary to covid-19. in total 137 regions were included in the study; italy (20) , usa (51), spain (17), france (22) and brazil (27) . running until the 17 may 2020, the following start dates and number of days are shown for each country: the search interest data with 7-day moving-mean, although continuous, would still have been bound at an upper and lower limit. in addition, kolmogorov-smirnov testing demonstrated the data was nonparametric. therefore, spearman's rank correlation test was used in all regions and for all 4 translated search concepts. daily search interest data (7-day moving-mean) were tested against daily new covid-19 cases per million (7-day moving-mean) over time within each region. for seven sets of results google trends returned an insufficient data response. this affected molise, basilicata, and valle d'aosta in italy and north dakota and wyoming in the usa. therefore, in all other combinations of regions and search concepts 541 spearman's rank correlation tests were performed table 2 shows a summary of the results. a strong (r s > 0.65) or moderate (r s 0.3 to 0.65) positive correlation between new daily covid-19 cases and search interest for 'loss of sense of smell' was observed in 110 regions (82%), for 'loss of sense of taste' 113 regions (84%), 'sense of smell' 107 regions (78%) and for 'sense of taste' 83 regions (61%). of this total of 413 strong to moderate test results, 381 had a significance value of less than 0.001. our data shows 121 regions (88%) as having a moderate or strong correlation between either 'loss of sense of smell' or 'loss of sense of taste' and new daily covid-19 cases. weakly positive or negative correlations (r s 0.3 -−0.3) were observed in 10 regions for 'loss of smell' (7.5%), 7 regions for 'loss of sense of taste' (5.1%), 12 regions for 'sense of smell' (8.8%) and 10 regions for 'sense of taste' (7.4%). moderate negative correlations (r s −0.3-0.65) were not observed in any regions for 'loss of sense of smell', 2 regions for 'loss of sense of taste' (1.5%), 3 regions for 'sense of smell' (2,2%) and 5 regions for 'sense of taste' (3.7%). no strongly negative correlations were observed. brazil had the highest number of strong or moderate correlations at 97 out of 108 (90%), then france with 78 out of 88 (89%), spain had 56 out of 68 (82%), italy 52 out of 75 (69%) and then the usa with 130 out of 202 (64%). the usa had a tendency toward more regions with moderate than strong correlations, whereas for regions from other countries the reverse is true. italy had the highest proportion of weak correlations with 8 (11%) and the usa the highest amount of moderate negative correlations (3.5%). our data indicate that in general for any individual region there is a temporal relationship between google search volume for terms relating to anosmia and ageusia and new cases of covid19 . this phenomenon appears strongest for search terms specifically mentioning loss of either smell or taste. weighted search interest scores were produced by applying daily regional weighting data to the original time series data. kolmogorov-smirnov testing indicated the weighted search table 2 . summary of spearman's rank correlation test outcomes for search interest in terms relating to anosmia and ageusia and new daily covid-19 cases per million (both data as 7-day moving-mean) the table shows counts of regions within each country and result group. r s : spearman's rank correlation coefficient. * significance level of p < 0.05. france 16 2 2 0 0 2 0 italy 11 5 2 0 0 2 0 usa 5 27 5 2 2 5 5 sense of taste brazil 19 8 0 0 0 0 0 spain 3 5 1 1 0 5 2 france 6 11 2 0 0 3 0 italy 3 4 1 2 2 2 6 usa 3 21 3 0 3 12 7 interest data pooled by country was not normally distributed. spearman's rank correlation analysis was used to assess the relationship between weighted search interest (7-day moving mean) and daily new covid-19 cases per million (7-day moving mean) over time and regions within each country. the results are shown in table 3 . all four search concepts demonstrated a significant (p < 0.05), positive correlation with new daily covid-19 cases. however, 'loss of sense of smell' and 'loss of sense of taste' had a higher range for r s when compared to 'sense of smell' and 'sense of taste'. this finding conforms to our previous observation that interest in terms including loss of either smell or taste more closely follow population changes in covid-19 cases. the most obvious differences between countries are that italy has a slightly lower range for the test statistic (r s 0.141 to 0.510) and brazil a higher range (r s 0.601 to 0.637). these differences appear reduced with search interest for 'loss of sense of smell' and 'loss of sense of taste'. the data suggest for any given point in time within a single country that regions with higher covid-19 cases will also have higher search interest for terms relating to anosmia and ageusia. within region analysis reveals that when considering a single region, higher numbers of covid-19 cases typically correspond with higher search interest for terms relating to anosmia and ageusia. between region analysis demonstrates an association between the relative increase in covid-19 cases and relative increase in search interest for anosmia and ageusia that shows consistency across all regions within a country. taken together this data is evidence of a widespread association between covid-19 cases and search interest for anosmia and ageusia that is observed in regional geographies inside the studied countries. as often stated, a global pandemic is a series of smaller national epidemics which themselves consist of a series of smaller outbreaks. walker, et al.'s findings demonstrate an association between search interest in loss of smell and new covid-19 cases at a national level and we have now found evidence of this at a regional level reflecting smaller disease outbreaks [8] . when analyzing whether the relative size of search interest changes reflected numbers of covid-19 cases between regions we were also able to demonstrate positive correlations. observed regional and national variations in the strength of the association we describe may be attributed to factors we could not account for. such factors would include differences in internet usage behaviors, the level public awareness of covid-19 symptoms and internet availability. as we now know the significance of the reduction of sense of smell in covid-19 positivity, finding ways of tracking the reporting of this symptom is important. mobile applicationbased reporting tools, such as the one produced by menni et al., could be one method of tracking cases based on symptomatology [22] . however, allowing access to geo-locations on a population wide level is controversial due to impacts on the right to privacy and as such may prove unpopular. tracking search terms does not identify an individual and does not interfere with this human right. tracking symptomatology through google is a newly developed technique and has transferability to other areas of medicine and future pandemics depending on the disease phenotype. due to the recent increase in widespread usage of mobile internet and the use of google on the biggest platforms as the primary search engine, this data has become more reliable when compared to previous years. comparison with previous pandemics, due to this behavioral change in internet interaction, was not undertaken in this study for that reason. covid-19 has affected numerous nations with significantly disparate gross domestic products (gdps) and public health budgets. sudan, for example, spends £25 per capita on health compared with £2989 in the uk [23, 24] . countries, which are not able to invest in pcr testing or future antibody assays, will look for more cost-effective means to track their outbreaks and reduce transmission. the use of mobile internet, in general, is high in these low-to-middle income countries and; 75% of the population of sudan have access [25] . excess mortality is now being adopted as the most accurate measure of covid-19-related mortality due to differences in testing between nations and the relatively low sensitivity of the standard pcr test [26, 27] . loss of smell search terms have the potential to identify active cases on an international basis as long as access to the internet and subsequent search term data is freely available. this is something that is more easily accessible and available in less developed countries with healthcare infrastructure that is not well resourced. we do not yet know if similar trends will continue in countries who have already been impacted by a large number of covid-19 cases but this can be assessed prospectively going forwards and modeling can be subsequently adapted. we have demonstrated that there is clear association between google trends search terms relating to loss of smell and taste and covid-19 cases both on a regional, national, and international basis. we feel that google search trends relating to loss of smell can table 3 . spearman's rank correlation test outcomes for weighted search interest in terms relating to anosmia and ageusia and new daily covid-19 cases per million (both data as 7-day moving-mean) or time and regional geography. r s = spearman's rank correlation coefficient. be utilized to identify potential covid-19 outbreaks on a regional basis within countries. this could help the implementation of targeted public health measures in these areas; which will be of particular benefit in low-to-middle income countries where testing is not widely available. if regional outbreaks are identified, spread throughout a country could be prevented whilst allowing other areas to continue with reduced restrictions therefore diminishing negative impact on economic growth and non-covid-19-related healthcare activity which is of significant concern in itself. https://github.com/georgecherry/anosmia_search_interest_covid-19 this paper was not funded. the authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. this includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties. peer reviewers on this manuscript have no relevant financial or other relationships to disclose. gc contributed to study design, developed the software which retrieved the data, analyzed the data and drafted the methods and results section of the manuscript. jr contributed to the study design and drafted the introduction, discussion, and conclusion sections of the manuscript. mc, jl, ml, vl, and bnk revised the manuscript and approved the final version for publication. papers of special note have been highlighted as either of interest (•) or of considerable interest (••) to readers isolated sudden onset anosmia in covid-19 infection. a novel syndrome? neurologic manifestations of hospitalized patients with coronavirus disease 2019 in wuhan, china objective olfactory evaluation of self-reported loss of smell in a case series of 86 covid-19 patients is loss of sense of smell as a diagnostic marker in covid-19: a systematic review and meta-analysis. authorea this systematic review, currently under peer-review, identified the positive predictive value for olfactory dysfunction and a positive covid-19 pcr swab. the studies included in the review agree that olfactory dysfunction is a key symptom in covid-19 patients and is more prevalent than cough and fever in this cohort scientific and ethical basis for social-distancing interventions against covid-19 how will country-based mitigation measures influence the course of the covid-19 epidemic? lancet covid-19 polymerase chain reaction testing before endoscopy: an economic analysis the use of google trends to investigate the loss of smell related searches during covid-19 outbreak. int forum allergy rhinol et al were the first group to investigate google trends on a national basis. they found positive correlations in search terms for loss of smell and covid-19 cases faq about google trends data -trends help this link explains how google trends data is collected and how the popularity of a search term is scored on a regional basis international organization for standardisation. iso online browsing platform (obp) the python package index. pytrends · pypi rome: dipartimento della protezione civile johns hopkins -centre for systems science and engineering madrid: centro nacional de epidemiología données des urgences hospitalières et de sos médecins relatives à l'épidémie de covid-19 brasilia: brasil pelo ministério da saúde rome: istituto nazionale di statistica instituto nacional de estadistica. resultados semestrales centro nacional de epidemiología paris: l'institut national de la statistique et des études économiques loss of smell and taste in combination with other symptoms is a strong predictor of covid-19 infection public financing for health in africa: from abuja to the sdgs how does uk healthcare spending compare with other countries? office for national statistics mobile cellular subscriptions (per 100 people) -egypt. arab rep detection of sars-cov-2 in different types of clinical specimens • wang et al demonstrate the relatively low sensitivity of the current covid-19 pcr test which reinforces the need for other strategies in tracking outbreaks other than testing alone covid-19: a need for real-time monitoring of weekly excess deaths key: cord-294955-bybdn9yb authors: brkic, faris f.; besser, gerold; janik, stefan; gadenstaetter, anselm j.; parzefall, thomas; riss, dominik; liu, david t. title: peaks in online inquiries into pharyngitis-related symptoms correspond with annual incidence rates date: 2020-09-23 journal: eur arch otorhinolaryngol doi: 10.1007/s00405-020-06362-4 sha: doc_id: 294955 cord_uid: bybdn9yb objective: to assess whether web-based public inquiries into pharyngitis-related search terms follow annual incidence peaks of acute pharyngitis in various countries from both hemispheres. methods: google trends (gt) was utilized for systematic acquisition of pharyngitis-related search terms (sore throat, cough, fever, cold). six countries from both hemispheres including four english (united kingdom, united states, canada, and australia) and two non-english speaking countries (austria and germany) were selected for further analysis. time series data on relative search interest for pharyngitis-related search terms, covering a timeframe between 2004 and 2019 were extracted. following reliability analysis using the intra-class correlation coefficient, the cosinor time series analysis was utilized to determine annual peaks in public-inquiries. results: the extracted datasets of gt proved to be highly reliable with correlation coefficients ranging from 0.83 to 1.0. graphical visualization showed annual seasonal peaks for pharyngitis-related search terms in all included countries. the cosinor time series analysis revealed these peaks to be statistically significant during winter months (all p < 0.001). conclusion: our study revealed seasonal variations for pharyngitis-related terms which corresponded to winter incidence peaks of acute pharyngitis. these results highlight the need for easily accessible information on diagnosis, therapy, and red-flag symptoms for this common disease. accurately informed patients might contribute to a reduction of unnecessary clinic visits and potentially cutback the futile antibiotic overuse. electronic supplementary material: the online version of this article (10.1007/s00405-020-06362-4) contains supplementary material, which is available to authorized users. acute pharyngitis is one of the most prevailing conditions in otorhinolaryngology with an annual prevalence of about 40% in pediatric patients and 15% in adults. the most commonly reported pharyngitis-related symptoms are sore throat [1] , followed by fever, headache, enlarged cervical lymph nodes, and cough [2] . acute pharyngitis is mostly caused by viral infections and often requires only symptomatic therapy [1] . yet, only 10-15% of cases are related to bacterial infections which warrant further antibiotic therapy [3] . nevertheless, overtreatment with antibiotics is common and frequently caused by misinformed patients hoping for rapid pain relief and fast recovery. this overtreatment and other reasons, such as availability of antibiotics without prescription, contribute to the global issue of antibiotic resistance [4] . moreover, acute pharyngitis also leads to a significant socioeconomic burden due to unnecessary ambulatory visits by misguided and misinformed patients [5] . the incidence of acute pharyngitis shows seasonal variations, with peaks during winter months. it may be caused by desiccated pharyngeal mucosa due to dry and cold air, making it friable and vulnerable to infections with the higher number of respiratory viruses during the cold season [3] . considering that the vast majority of teenagers and adults use the world wide web to acquire health-related information, we hypothesized that peaks in web-based internet searches for pharyngitis-related symptoms might also follow the global incidence rates of this condition. however, up to now, there is no literature available addressing this matter. the internet has become the most important information source for people generally, and particularly in cases of health-and disease-related issues [6] . the most popular online search engine is google, with about 70% of daily world wide web searches performed using this platform [7] . google trends (gt) is a publicly available analysis tool that allows a keyword-driven analysis of a portion of google searches performed [8] . infodemiology is a newly proposed research area. it involves the analysis of online search patterns to gain more insight on human behavior to inform medical professionals [9] . indeed, several medical reports assessed the seasonality of different symptoms and diseases, such as tinnitus, epistaxis, laryngitis or dengue fever using gt [9] [10] [11] [12] [13] [14] [15] . however, up to now, literature has been sparse on web-based inquiries regarding acute pharyngitis or its most common symptoms. therefore, the aim of this study was to assess web-based public interest for acute pharyngitis and related-terms for seasonal variations globally. furthermore, the validity and reproducibility of data acquired using gt was analyzed. the results are discussed in terms of the necessity for clear, easily accessible, and accurate general information on acute pharyngitis. gt (google llc) is an online, publicly accessible, search term analysis engine. it allows analysis of search-query volume (frequency) for search terms that were entered on google web search. these searches can be grouped for geographical location, timeframe (dating back to 2004), category, and subject [9, 10] . the search frequency is displayed as the relative search volume (rsv), indicating the interest for a specified search-term. furthermore, gt allows comparing the rsv of up to five different search-terms. moreover, the "related queries" option of gt allows the exploration of search terms that users entered after searching for the targeted keyword [9] . to assess and illustratively depict seasonal variations of global rsv for pharyngitis-related search terms, we included countries from both hemispheres. in line with previous studies, we selected australia, canada, the united kingdom (uk), and the united states of america (usa) as english-speaking countries from both hemispheres [9, 10] . we selected germany and austria as non-english speaking countries from the northern hemisphere. to explore pharyngitis related search terms that google users entered on google web search to gain more insight into acute pharyngitis, we entered five different search terms related to pharyngitis and its country-specific translations on april 22nd 2020. . we then applied gt-option "related queries" for each of the above-mentioned keywords and noted all related inquiries (supplementary table 1 ). in addition, we compared all related inquiries with one another to depict the five most relevant search terms. since previous studies provided evidence that gt-data from countries with a lower number of inhabitants are less reliable [9, 10] , we entered the five above-mentioned search terms on seven consecutive days for each country included, starting from april 22nd 2020. gt searches were specified for the following parameters: timeframe between january 1st, 2004 until december 31st, 2019 and "health" category. for interpretation of seasonal patterns, winter months of the northern hemisphere were defined as january, february, and march, while summer months were defined as june, july, august, and september. winter and summer months were defined vice versa for the southern hemisphere. data were analyzed using the "season" and "psych" package in r 3.5.1 (r development core team, 2008; r foundation for statistical computing, vienna, austria). following graphical visualization based on histograms, single-time series data were analyzed for reliability using the intraclass correlation coefficient (icc 2,1 ) [16] . subsequently, annual seasonal variations in rsv were assessed for data retrieved from the first day of data extraction (april 22nd 2020) of time series data using the cosinor analysis. the exact model is described in more detail elsewhere [17] . summed up, the cosinor model fits a sine wave to a predefined timeframe based on linear regression. since we assessed annual variations, one peak was defined for every 12 months. the sinusoid is characterized based on a phase (p, peak) and an amplitude (a, size of the peak). statistical significance of the model was tested with alpha level set at 0.025 to control for type i errors. first, we determined the most relevant pharyngitis-related search terms in english and non-english speaking countries from both hemispheres. we, therefore, applied gt-function "related queries" and compared all pharyngitis-related inquiries with one another to explore the most relevant pharyngitis related search terms. the analyses revealed different relevant search terms in english and non-english speaking countries, which were used for subsequent analysis. we have previously shown that data from countries with less inhabitants show lower reliability compared to higher populated countries [9, 10] . we aimed to determine the reliability of gt-data which were assessed on seven consecutive days as mentioned above using the intraclass correlation coefficient (tables 1, 2) (supplementary tables 2-5). analysis revealed good to excellent reliability for all search terms. search terms from countries with lower number of inhabitants such as australia or austria showed a slightly lower reliability compared to larger countries such as the usa or germany. we then sought to determine the peak month for public interest in sore throat-related search terms for single time series data queried from the first day of data-retrieval-april 22nd, 2020 (tables 3, 4; fig. 1 ). cosinor analysis revealed significant seasonal variations and peaks in winter months for all search terms of countries from both hemispheres. for english speaking countries of the northern hemisphere fig. 1f ). acute pharyngitis is mostly caused by viral infections, which require only symptomatic therapy. however, some cases warrant antibiotic therapy due to a bacterial infection. the differentiation between these two conditions is fairly difficult for non-medical professionals, leading to a frequent over-treatment with antibiotics. this substantially contributes to the fact that acute pharyngitis is a significant burden on health care systems worldwide [5] . these issues highlight the need to assess web-based public inquiries into pharyngitis to improve the effectiveness of web-based information distribution to the general population. the current study revealed winter peaks in world wide web inquiries for pharyngitis-related terms in countries from both hemispheres. these peaks correspond to annual incidence cycles of acute pharyngitis [3] . moreover, the analysis also revealed excellent reliability for gt-inquiries into pharyngitis-related search terms. as mentioned above, an estimated 80% of acute pharyngitis cases are of viral etiology and require only symptomatic therapy. this symptomatic therapy includes systemic and local analgesic and anti-inflammatory medication. yet, the use of antibiotics remains high, although often not necessary and inappropriate [4] . regarding therapy options for acute pharyngitis, new treatment regimens were investigated recently. essak et al. [18] analyzed the use of topical antibiotics, which resulted in no clear benefits. another group investigated the use of local, low dose flurbiprofen for the management of acute sore throat [19] . they concluded that this treatment option represents a useful first-line therapy for symptomatic relief of patients with acute pharyngitis/sore throat. in addition, the authors proposed that this new therapy option may lead to a reduction of unnecessary antibiotic prescriptions. treatment-related information can be easily obtained on the world wide web by medical professionals. however, patients that try to treat themselves might easily be overwhelmed by the wealth of information that is available online. therefore, it is crucial to provide reliable, easily accessible, and publicly available online information on diagnosis, treatment, and red-flag symptoms of usually self-limiting medical conditions such as acute pharyngitis. for example, vital information on pharyngitis related red-flags requiring special attention and therapy would include worsening symptoms or (unilateral) neck swelling or trismus as a possible sign of tonsillitis with beginning complications [20] . as mentioned above, patients looking for rapid recovery and pain relief tend to self-manage their complaints inappropriately [4] . due to high incidence rates, this could certainly contribute substantially to antibiotic overuse and resistance. moreover, the phenomenon of prescription-free antibiotics was noted, which again supports patient's self-care and self-(mis)management [21] . purchasing prescription antibiotics from other countries is an already widely discussed topic [22] . lower thresholds for prescribing antibiotics as well as issuing them without medical assessment promotes self-treatment even further [23] . moreover, there are several online pharmacies registered in the uk that do not require prescriptions for antibiotics, some of them even allowing patient-driven decision for antibiotic dose, choice and quantity [24] . the so-called no-prescription websites certainly reflect a further negative side of the world wide web [25] . these aggravate the problem of acquiring prescriptionfree medication including antibiotics. based on the abovementioned issues, the importance of properly informed general public is further underlined, as well as the monitoring of these online services by law enforcement agencies and health initiatives. acute pharyngitis was the cause for about 13 million outpatient visits in a single year in the usa [26] . the economic burden of therapy and doctors' visits are, therefore, enormous with annual costs of approximately 1.2 billion us dollars in the usa [5] . in contrast, annual healthcare costs in the usa for the much more prevalent and potentially more severe influenza type a and b infections ranged from 2 to 5.8 billion us dollars [27] . a possible cost reduction of 30% was estimated with reduced unnecessary visits to outpatient departments and further 7.3% when cutting the futile antibiotic therapy for patients with an acute pharyngitis [5] . until now, no clear guidelines exist in regard to diagnostics and therapy of acute pharyngitis. however, european guidelines consider it a self-limiting disease that does not automatically require specific therapy except for symptomatic. antibiotic therapy is recommended only in highrisk patients [28, 29] . the importance of patients being accurately informed about medical conditions has already been discussed extensively. van der velden [30] provided a new structured approach for the management of an acute sore throat. the authors noted that informed and educated patients are a significant step for empowering self-management. first, patient's expectations and concerns as well as their opinion on antibiotic use should be identified. second, the severity of the condition must be assessed. this involves identifying risk factors for adverse events, e.g., red flags, such as unilateral neck swelling or trismus for patients with sore throat/pharyngitis [20] . indeed, one randomized controlled trial showed that well-tailored websites with information on basic medical conditions such as sore throat, fever, runny nose etc. led to a better understanding of these problems by the general public compared to standard online-available information [31] . furthermore, they provided first evidence that this type of online information service can contribute to self-managing of minor medical symptoms. besides common pharyngitis symptoms such as sore throat, cough and fever [1, 2] , we also included other search terms during the analysis. common cold is an often-used umbrella term for acute infections of the upper respiratory tract including acute pharyngitis [32] . similarly, incidence peaks for these medical conditions occur mostly during the winter months [33] . based on this consideration, the search term "cold" was included in our analysis. likewise, the search term "erkältung" was included in the analysis of german-speaking countries. furthermore, given the fact that viral pharyngitis is often misidentified and mistreated as a streptococcal tonsillitis [4] , we included the term "strep" in our analysis. in recent years, gt was often used for infodemiological assessment of different medical phenomena [9] [10] [11] [12] [13] [14] [15] . as shown by kang et al. [34] , gt has been proven to be a good complementary source for surveillance of annual influenza outbreaks and noted that gt could be used for detecting early signals of major outbreaks. a raised concern is, however, the lack of reliability assessment in these studies. searches using gt delivered slightly different results when performed on different timepoints (e.g., days), which was already reported in another study [16] . we have previously shown that the number of inhabitants of assessed countries might be related to the reliability of gt searches. to overcome the bias in this regard, we performed gt searches on 7 consecutive days. we were able to show excellent reliability for gt-inquiries on pharyngitis-related search terms for all included countries. moreover, our analysis provided further evidence that gt-data from larger countries were more reliable. this was in alignment to results of previous studies [9, 10] . again, we point out the importance of reliability assessment of data retrieved using gt. the recent covid-19 pandemic led to a significant rise in online inquiries related to covid-19 and related symptoms [35] [36] [37] . public inquiries into coronavirus has already been assessed using gt [37] . it was demonstrated that the use of digital epidemiology (e.g., infodemiology) allows good surveillance of local covid-19 outbreaks. to circumvent the bias regarding overlapping symptoms to those of an acute pharyngitis (sore throat, cough, fever etc. [38] ), we did not perform the search beyond december 31st 2019. telephone triage and phone consultations have gained increasing importance during the current covid-19 pandemic [39] . in case of acute pharyngitis, patients could be informed on red flags and symptoms of acute bacterial tonsillitis or signs of a peritonsillar abscess, which would require further medical assessment. furthermore, pharmacists may play an important role in providing patients with basic medical information and directing them to health care professionals when necessary [40] . this is crucial particularly for patients trying to self-manage certain medical conditions which tend to visit pharmacies looking for pain killers or antibiotics. furthermore, pharmacists might also represent the only medical contact person for these patients, as many would not visit a doctor's office. despite new findings of the current study, there are some limitations that warrant further discussion. first, the data was searched using only one online engine (gt), which could have resulted in a selection bias; this risk is, however, minimized by the fact that almost 70% of online searches are performed using google [7] . second, no demographic data such as gender or age of users as potential covariates are included in the data. it has been noted that younger individuals tend to perform online searches regarding medical conditions more frequently than older adults do [41] . this may have an influence on the probability of looking up symptoms, diagnosis or therapy based on online search engines. however, when compared to epidemiological studies, data gathered using infodemiological methods are more real time, extensive, and include area span and time. furthermore, infodemiological methods also simplify the process of information retrieval and improve the efficiency of research. we revealed seasonal variations in world wide web-based inquiries for pharyngitis-related terms, which correspond to annual incidence rates of the acute pharyngitis. these findings underline the importance of accurate and easily accessible medical online information, in particular information regarding treatment options, diagnostic algorithms, and redflag symptoms. this may represent a significant step towards the reduction of unnecessary clinic visits. moreover, the antibiotic overuse could be scaled down, therefore, contributing to a reduction of individual and global antibiotic resistances. funding open access funding provided by medical university of vienna. code availability not applicable. conflict of interest no conflicts of interest or disclosures are declared by authors of the study. ethics approval we conducted this study using non-personally identifiable information and entirely publicly accessible data. therefore, the approval of an ethics committee was not needed. consent for publication not applicable. open access this article is licensed under a creative commons attribution 4.0 international license, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the creative commons licence, and indicate if changes were made. the images or other third party material in this article are included in the article's creative commons licence, unless indicated otherwise in a credit line to the material. if material is not included in the article's creative commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. to view a copy of this licence, visit http://creat iveco mmons .org/licen ses/by/4.0/. sore throat streptococcal acute pharyngitis common questions about streptococcal pharyngitis streptococcal pharyngitis in children: to treat or not to treat? economic burden of adult pharyngitis: the payer's perspective online health information seeking among us adults: measuring progress toward a healthy people 2020 objective seasonality of cellulitis: evidence from google trends riss d (2020) seasonal variations in public inquiries into laryngitis: an infodemiology study winter peaks in web-based public inquiry into epistaxis frequency and seasonal variation of ophthalmology-related internet searches seasonal trends in tinnitus symptomatology: evidence from internet search engine query data correlation between google trends on dengue fever and national surveillance report in indonesia using google trends to assess global public interest in osteoarthritis using google trends to understand information-seeking behaviour about throat cancer low validity of google trends for behavioral forecasting of national suicide rates analysing seasonal data topical (local) antibiotics for respiratory infections with sore throat: an antibiotic stewardship perspective locally delivered flurbiprofen 8.75 mg for treatment and prevention of sore throat: a narrative review of clinical studies expand the pharyngitis paradigm for adolescents and young adults do free or lowcost antibiotic promotions alter prescription filling habits? socioeconomic and demographic characteristics of us adults who purchase prescription drugs from other countries is it getting easier to obtain antibiotics in the uk? obtaining antibiotics online from within the uk: a cross-sectional study observation of a pharmacist-conducted group a streptococcal pharyngitis point-of-care test: a time and motion study us healthcare costs attributable to type a and type b influenza agoritsas t (2017) corticosteroids for treatment of sore throat: systematic review and meta-analysis of randomised trials management of acute streptococcal pharyngitis: still the subject of controversy sore throat: effective communication delivers improved diagnosis, enhanced self-care and more rational use of antibiotics evaluation of a web-based intervention providing tailored advice for self-management of minor respiratory symptoms: exploratory randomized controlled trial common cold -an umbrella term for acute infections of nose, throat, larynx and bronchi probiotic effects on cold and influenza-like symptom incidence and duration in children using google trends for influenza surveillance in south china the use of google trends to investigate the loss ofsmell related searches during covid-19 outbreak association of the covid-19 pandemic with internet search volumes: a google trends tm analysis correlations of online search engine trends with coronavirus disease (covid-19) incidence: infodemiology study recommendations for management of acute pharyngitis in adults impact of the covid-19 pandemic on the core functions of primary care: will the cure be worse than the disease? a qualitative interview study in flemish gps the role of pharmacists in patients' education on medication let me google that for you: a time series analysis of seasonality in internet search trends for terms related to foot and ankle pain key: cord-018688-gvk9uazp authors: magid, avi; gesser-edelsburg, anat; green, manfred s. title: the role of informal digital surveillance systems before, during and after infectious disease outbreaks: a critical analysis date: 2018-03-23 journal: defence against bioterrorism doi: 10.1007/978-94-024-1263-5_14 sha: doc_id: 18688 cord_uid: gvk9uazp background one of the main limitations of traditional surveillance systems for disease detection is their inability to detect epidemics in real-time. in addition to syndromic surveillance, a number of informal digital resources have been developed. these systems are based on data collected through media sources such as news reports on the internet, mailing lists, and rss (really simple syndication) feeds. the role of such systems at all stages of the epidemic remains unclear. methods a literature review was carried out on informal digital resources for infectious disease surveillance. we examined the source of information, the manner in which they process and disseminate the information, their role in each phase of disease outbreaks, and whether and to what extent these systems are capable of early detection and management of infectious disease epidemics. results informal digital resources use similar sources of data for surveillance. however, they use different algorithms to create their output, and cover different geographic areas. in this regard, they complement each other with respect to information completeness. there is evidence in the literature on the systems’ usefulness in communicating information to public health professionals, as well as to the general public during and after previous epidemics. retrospective studies of some systems have shown a theoretical decrease in the time of epidemic detection compared to conventional surveillance. however, there is no evidence of the ability for real-time detection. conclusions currently, there is little prospective evidence that existing informal systems are capable of real-time early detection of disease outbreaks. most systems accumulate large amounts of information on a wide variety of diseases, making it difficult to extract critical information. presenting critical information clearly and precisely remains a challenge. not be captured through traditional surveillance, and may be useful to governments and health agencies [4] . these systems are designed to function during all phases of disease outbreak, and are planned to increase sensitivity and timeliness. however, the role of such systems before, during and after infectious disease epidemics and, in particular, whether such systems are currently capable of early detection of epidemics remains unclear. a literature review was carried out to compare informal digital systems with regards to their source of information, the manner in which they process and disseminate the information, their role in each phase of an epidemic, and whether and to what extent these systems are capable of early detection of epidemics. the systems evaluated were promed-mail, global public health intelligence network (gphin), healthmap, medisys, epispider, biocaster, h5n1 google earth mashup, avian influenza daily digest and blog, google flu trends and argus. promed-mail is "an internet based reporting system aimed at rapidly disseminating information on infectious disease outbreaks and acute exposures to toxins that affect human health, including those in animals and in plants grown for food or animal feed" [21] (promed-mail website). promed-mail receives information from a number of sources, such as media reports, official reports, online summaries and local observers. the reports are reviewed and investigated by promed-mail expert team, and then distributed by e-mail to promed subscribers, and published in promed-mail website (promed-mail website). in addition to filtering the received information, promed-mail expert team may also add related information from media, government and other sources [23] . promed-mail was proven as an efficient system during the 2003 outbreak of sars, where information about points of outbreak, including additional information from a british medical journal article, was efficiently disseminated [23] . it should be stressed that promed-mail collects, filters, disseminates and archives it. they do not carry out formal analysis of the information although they provide some evaluation. the global public health intelligence network (gphin) is a biosurveillance system developed by health canada in collaboration with the who. gphin receives as input, information about disease outbreaks arriving from news service items, promed-mail, electronic discussion groups and selected websites, and disseminates information to subscribers using the following decision algorithm. a relevance score is computed for each information item. two thresholds are determined, high and low. if the item relevance score is greater than the high threshold, then it is immediately disseminated to subscribers. if the item relevance score is lower than the low threshold, then it is automatically "trashed". otherwise (if the item relevance score is between the high and the low thresholds), the item goes through human analysis and then disseminated to subscribers [23] . a prominent limitation of gphin efficiency is its reliance on the time in which information about an outbreak or other event if published in one of gphin data sources. nevertheless, gphin is considered efficient in providing earlier warning of events of interest to the international community compared with other systems, as 56% of the 578 outbreaks verified by who between july 1998 and august 2001 were initially picked up by gphin [23] . healthmap is a freely accessible automated electronic information system aimed at facilitating knowledge management and early detection of infectious disease outbreaks by aggregating, extracting, categorizing, filtering and integrating reports on new and ongoing infectious disease outbreaks. data on outbreaks are organized according to geography, time, and infectious disease agent [5] . healthmap receives as input reports received from variety of electronic sources, including online news sources aggregated in websites such as google news, reporting systems such as promed-mail, and validated official reports received from organizations such as the who [5, 11] . an internet search is performed by healthmap every hour, 24 h a day, in order to obtain the required information. search criteria include disease name (scientific and common), symptoms, keywords, and phrases. after collecting the reports, healthmap uses text mining algorithms in order to characterize the reports. characterization includes the following stages: (1) categorization: reports are categorized according to disease and location and relevance is determined. (2) clustering: similar reports are grouped together and exact duplicates are removed. clustering is performed based on similarity of the report's headline, body text, and disease and location categories. (3) filtering: reports are reviewed and corrected by an analyst, and then filtered into five categories -breaking news, warning, old news, context, and not disease related. in order to reduce information overload and to focus on disseminating information regarding outbreaks of high impact, only reports classified as breaking news are overlaid on an interactive geographic map located on healthmap site [5] . among the users of healthmap are the who, the us centers for disease control and prevention, and the european center for disease prevention and control, which use its information for surveillance activities [5, 11] . medical information system (medisys) is an informal automatic public health surveillance system. medisys is designed and operated by the joint research center (jrc) of the european commission, in cooperation with the health threat unit at the european union directorate general for health and consumer affairs and the university of helsinki. medisys collects its information from open-source news media, mainly articles from news pages. medisys categorizes the collected information according to predefined categories and disseminates it to subscribed users by e-mail. the system also provides its user with features and statistics available on its website, including a world map in which event locations are highlighted, aggregated news count per each geographic location presented on graphs, and the most significant event location for the last 25 h. medisys is available in 26 languages (the system collects information in 45 languages, but the website is available in 26 languages). users can filter the information according to language, disease and location, as well as by outbreaks, treatments and legislations. medisys users can also select articles into predefined categories, add comments to these articles, add information, and disseminate them to user-defined groups [12] . argus is an informal biosurveillance system aimed at detecting and monitoring biological events that may be a global health threat to human, plant and animals. the system is hosted at the georgetown university medical center (washington, dc, united states), and funded by the united states government. argus collects information in 40 native languages from media sources, including printed newspapers, electronic media, internet-based newsletters and blogs, as well as from official sources (the world health organization (who) and the world organization for animal health (oie). the system uses bayesian analysis tools for selecting and filtering the collected articles. the process is performed by about 40 regional professional analysts, who monitor several thousand internet sources on a daily basis. by using bayesian analysis tools, the analysts select reports from a dynamic database of media reports. relevance is determined according to a specific set of terms and keywords applicable to infectious diseases surveillance. after filtering the information, events that may indicate the initiation of an outbreak are disseminated to the system users. also disseminated are events that may require investigation [12, 20] . biocaster is an informal surveillance system aimed to collect information on disease outbreaks, filter the information, and disseminate it to users. the system is a part of a research project developed and managed by the national institute of informatics in japan, which involves five institutes in three countries. biocaster focuses mainly on the asia-pacific region. the system collects information by using really system syndication (rss) feeds from more than 1700 sources. information is collected mainly from google news, yahoo! news, and european media monitor, filtered and disseminated in a fully automated manner with no human analysis in any stage. filtered information (about 90 articles per day) is published in three languages (english, japanese and vietnamese). articles are processed and disseminated every hour. in addition, biocaster creates an ontology which covers approximately 117 infectious diseases and six syndromes. the ontology is produced in eight languages (english, japanese, vietnamese, chinese, thai, korean, spanish and french), and is used as an input to global health monitor web portal, which offers its users maps and graphs of health-concerning events [12] . the semantic processing and integration of distributed electronic resources for epidemiology (epispider) is a web-based tool which integrates information gathered from electronic media resources containing health information, as well as from informal surveillance systems, such as promed-mail. the aim is to enhance the surveillance of infectious disease outbreaks.epispider uses promed-mail reports as an input, as well as health news sources that provide rss feeds. by using natural language processing, it extracts location information from the input sources, and geocode them using the yahoo and google geocoding services. after a filtering process, the system generates summaries of promed reports (on a daily basis). these reports are available in the epispider website [13] . google earth combines satellite images, aerial photography and map data to create a 3d interactive template of the world. this template can be used by anyone to add and share information about any subject that involves geographical elements. nature (international weekly journal of science) uses google earth to track the spread of the h5n1 avian flu virus around the globe, and to present a geographic visualization of the spread of h5n1 [19] (nature website). avian influenza daily digest is a digest produced by the united states government. the digest collects raw open source content regarding avian influenza and disseminates it to subscribers. material is disseminated without any processing. users are encouraged to provide with updates and/or clarifications that will be posted in subsequent issues of the digest [2] . google flu trend is designed by google internet company to be a near real-time tool for detection of influenza outbreaks. google flu trend exploits the fact that millions of people worldwide search online for health-related information on a daily basis. the tool was designed based on the assumption that there is an association between the number of people searching for influenza-related topics and the number of people who actually have influenza symptoms, and therefore, an unusual increase in the number of people searching for influenza-related topic on the web may simulate an increase in influenza syndromes. studies performed by google and yahoo have shown that plotting data on searches using influenza-related keywords has led to an epidemic curve that closely matched the epidemic curve generated by traditional surveillance of influenza [4] . google flu trends analyzes a fraction of the total google searches over a period of time, and extrapolates the data to estimate the search volume. the information is displayed in a graph called "search volume index graph". it is claimed by the tool's designers that, according to tool testing, it can detect outbreaks of influenza 7-10 days before it is detected by conventional cdc surveillance [4] . all the studied digital resources use similar sources of data -official reports, as well as media reports, including global media resources, news aggregators, eyewitness reports, internet-based newsletters and blogs. however, they use different algorithms to create their output, and cover different geographic areas. in addition, existing digital resources are different in the manner they filter and analyze the information and may create different output. therefore they complement each other with respect to information completeness. retrospective studies of some systems have shown a theoretical decrease in the time of outbreak detection compared to conventional surveillance. however, evidence of such ability in real time is sparse and unclear. chan et al. [8] have analyzed the average interval between the estimated start of the outbreak to the earliest date of discovery and publication, using who confirmed outbreak reports, as well as promed-mail, gphin and healthmap reports. analysis showed a decrease in intervals over 14 years, which was partially attributed to the emergence of informal digital resources [8] . a retrospective study of argus reports on respiratory disease in mexico showed a significant increase in reporting frequency during the 2008-2009 influenza season relative to that of 2007-2008. the authors suggest that, according to these retrospective results, respiratory disease was prevalent in mexico and reported as unusual much earlier than when the h1n1 pandemic virus was formally identified. however, its connection with the 2009 pandemic is unclear [20] . the google flu trends tool was also retrospectively tested. according to retrospective testing, influenza epidemics can be detected by using google flu trends tool 7-10 days before it is detected by conventional surveillance [6, 7] , however, there are still no prospective evidence to such capability. a retrospective study from china reported that google flu trend search data are correlated with traditional methods of surveillance [14] . another retrospective study tested the real-time detection ability of six informal digital systems, including argus, biocaster, gphin, healthmap, medisys and promed-mail. data from these systems were used to detect epidemics and compared to official data. results suggested that all tested systems have shown retrospective real-time detection ability. moreover, it was found that the combined expertise amongst systems provided a better early detection [3] . unlike retrospective evidence, prospective evidence of informal digital systems capability for early detection of epidemics is sparse. some epidemics have been claimed to be first reported by promed-mail, before they were officially reported by the who [15] . these reports were proved to be reliable, since they were later confirmed by the who. however most of the reports were first published by promed-mail not because the information was not available to the who by this time, but because the who was not authorized to publish them due to lack of conformation [15] . the sars in china (february 10, 2003) is the best known outbreak first reported on promed-mail [17] . a detection in real-time was also demonstrated by gphin during the sars outbreak of 2002. gphin detected sars and issues the first alert to the who more than 2 months before it was first published by the who [16, 18, 24] . however, the time between the gphin alert and the first time it was reported by the who implies that the whole detection process was not shortened due to the gphin alert. retrospective reviews of the polio outbreak of 2013 and 2014 and the ebola outbreak of 2014 showed that informal digital detection preceded official detection by an average of 14.6 days. for example, promed and gphin reported the polio epidemic in cameroon in 2013 23 days after the outbreak began, where the official who report was published 51 days after the outbreak began [1] . however, the digital systems detection did not contribute in real-time to the whole process of outbreak detection and declaration. hence, in real-time it is not an early detection. there is evidence in the literature on the systems' usefulness in communicating the information during previous outbreaks to public health professionals, as well as to the general public. promed-mail and gphin had critical roles in updating public health officials about the sars outbreak in 2002 [4] . such systems are also capable of providing officials, clinicians and the general public with guidance to medical decision making, including the importance of vaccination and other preventive actions [4] . the first report on sars on february 10, 2003 published by promedmail, and the hundreds of subsequent promed-mail reports have helped health professionals worldwide to gather critical details regarding sars, and by this to recognize sars and discover its cause [15] . assessment of correlation between healthmap reports and official government reports reported during the first 100 day of the 2010 haitian cholera outbreak has confirmed that data yielded from informal digital systems were well correlated with data officially reported from the haitian health authorities. moreover, this study has shown that informal digital systems are capable of being used at the early stages of an outbreak not only as an indicator of the outbreak occurrence, but also as a predictive tool by providing a reliable estimation of the reproductive number, a major epidemic parameter [9] . there is no evidence in the literature of the use of informal digital systems after an epiodemic. nevertheless, we believe that data collected during outbreaks through informal digital systems are being used by public health agencies for retrospectively studying the dynamics of epidemic, and for drawing conclusions about the management of the epidemic. there has been impressive progress in the development of informal digital systems for disease surveillance. informal digital systems are widely used by the general public, as well as by health officials. a good example is the goran digital system (the global outbreak and response network) developed by the who, which gather information from number of sources both governmental and informal, including gphin and promed-mail [17] . one of the most prominent suggested advantages of the digital systems is their functioning in early notification of infectious disease outbreaks, before the official notifications, and their contribution to the epidemiological investigation of the disease before official data are available. during epidemics, data gathered and disseminated through official public health authorities are usually not available to public health officials and to policy makers for some time, sometimes due to political and logistic limitations. this period of time is critical for estimating the epidemic dynamics and implementing the response plan [9] . unlike official data, data collected by digital systems are available in near real-time, and may be used for epidemiological assessment. a mandatory requisite for the use of digital systems data for epidemiological investigation of an outbreak is the reliability of the data, as well as their equivalence to official data. in other words, there should be a match between the number of cases derived from the informal data and the number of cases officially reported by public health authorities. indeed, our results have pointed out an example in which a correlation between digital systems data and official data in the first stages of epidemic was confirmed in the data collected from healthmap regarding the 2010 haitian cholera. however, as mentioned by the authors, epidemiological measurements using digital systems data should be also tested in other epidemics, in order to confirm the method's reliability [9] . the fact that the number of subscribers to digital systems is increasing each year [15] makes these systems an efficient tool for globally spreading the information, as well as a tool for epidemiological investigation, complementary to official data. however, despite their theoretical advantage over traditional surveillance, there is no evidence in the literature that information collected through digital system had affected public health policy makers. although we did not find evidence in the literature, we believe that digital systems may also contribute to the public health community after the outbreak ends. the abundance of reports collected and disseminated by these systems during outbreaks creates an epidemiological reservoir, which, due to its availability worldwide, may be used for a post-pandemic investigation and conclusion making. as for early detection of infectious disease outbreaks, we did not find any prospective evidence showing the capability of digital systems of detection infectious disease outbreaks in real-time. our results are consistent with some other studies conclusions, pointing out that currently digital systems are not capable of detecting an outbreak [17, 4] . although there is evidence of informal digital systems publishing reports on outbreaks before official detection (such as in the polio outbreak of 2013 and 2014 pointed out in the results section) [1] , these reports did not actually affect the process of detection. the formal process of detection includes receiving the information, processing the information and using the information. the early digital systems reports were not used in any of the detection phases and did not change the process. it may be viewed as an analogue to screening tests which are effective only if they are capable of changing the natural history of a disease. since there is no evidence of informal digital systems capable of changing the "natural history of outbreak" so far, they cannot be considered useful for early detection. informal digital systems may also have an important role in disease surveillance. incorporating informal digital systems into existing formal systems may improve their performance. a study in the united states showed that combining information gathered from informal digital systems with information received from the texas influenza-like-illness surveillance network (ilinet) improved the ability of predicting hospitalizations due to influenza [22] . another study in the united states showed a good correlation between google flu searches and emergency department influenza-like illness visits [10] . moreover, since digital sources usually contains data not captured through traditional methods, they are used by public health organizations, including the global outbreak alert and response network of the who, which uses digital sources for surveillance on a daily basis [4] . however, the usage of digital systems as a surveillance tool may have some limitations. first, most systems accumulate a huge mass of information on a large variety of diseases, making it difficult to extract critical information. in other words, no integration of the information is performed to yield useful information. the challenge is to present critical information clearly and concisely. second, digital systems are less specific than traditional surveillance systems, mostly due to false alarms, misinformation and information based on rumors [9, 17] . therefore, they may not be solely used but as a complementary tool for traditional surveillance systems [17] . a third limitation is the lack of a response system to early warnings. with the lack of such a system, early warning is not useful, as no practical action is followed by the publication of the information. such a response system may include triggers and decision criteria, which would lead to an appropriate and proportionate response to the threat [17] . to summarize, considerable efforts and resources have been invested in the development of informal digital system for detection of infectious disease outbreaks. as a result, a new generation of informal digital systems has emerged. the most prominent advantage of such systems is their ability to report on an outbreak in near real-time, or, in other words, before the information is officially reported, and by this to be used by public health decision makers for epidemiological assessment and preparation for the pandemic. currently there is no evidence in the literature for their capability to detect an outbreak at its onset. in addition, there are no hard data to prove the benefits of using such systems before and during an outbreak. we do not believe that they can be used to identify early cases, but should be used as a support system for describing the spread of the disease. the challenge is to empirically assess the efficiency of informal digital systems and their use for decision making and interventions during crisis, as well as to test the systems' sensitivity and specificity. a more general informal system, which provides syndromic-based analysis of reports disseminated by all currently existing systems, may be the next step toward disease outbreak detection based on informal systems. digital surveillance for enhanced detection and response to outbreaks factors influencing performance of internet-based biosurveillance systems used in epidemic intelligence for early detection of infectious diseases outbreaks digital disease detection -harnessing the web for public health surveillance surveillance sans frontieres: internetbased emerging infectious disease intelligence and the healthmap project google trends: a web-based tool for real-time surveillance of disease outbreaks google trends: a web-based tool for real-time surveillance of disease outbreaks global capacity for emerging infectious disease detection social and news media anables estimation of epidemiological patterns early in the 2010 haitian cholera outbreak google flu trends: correlation with emergency department influenza rates and crowding metrics healthmap: global infectious disease monitoring through automated classification and visualization of internet media reports landscape of international eventbased biosurveillance scanning the emerging infectious diseases horizon-visualizing promed emails using epispider using google trends for influenza surveillance in south china the internet and the global monitoring of emerging diseases: lessons from the first 10 years of promed-mail internet-based surveillance systems for monitoring emerging infectious diseases public health surveillance and infectious disease detection the global public health intelligence network and early warning outbreak detection: a canadian contribution to global public health event-based biosurveillance of respiratory disease in mexico optimizing provider recruitment for influenza surveillance networks book: handbook of biosurveillance 301 early detection of disease outbreaks using the internet key: cord-018632-azrqz6hf authors: ganasegeran, kurubaran; abdulrahman, surajudeen abiola title: artificial intelligence applications in tracking health behaviors during disease epidemics date: 2019-11-21 journal: human behaviour analysis using intelligent systems doi: 10.1007/978-3-030-35139-7_7 sha: doc_id: 18632 cord_uid: azrqz6hf the threat of emerging and re-emerging infectious diseases to global population health remains significantly enormous, and the pandemic preparedness capabilities necessary to confront such threats must be of greater potency. artificial intelligence (ai) offers new hope in not only effectively pre-empting, preventing and combating the threats of infectious disease epidemics, but also facilitating the understanding of health-seeking behaviors and public emotions during epidemics. from a systems-thinking perspective, and in today’s world of seamless boundaries and global interconnectivity, ai offers enormous potential for public health practitioners and policy makers to revolutionize healthcare and population health through focussed, context-specific interventions that promote cost-savings on therapeutic care, expand access to health information and services, and enhance individual responsibility for their health and well-being. this chapter systematically appraises the dawn of ai technology towards empowering population health to combat the rise of infectious disease epidemics. infectious diseases disrespect national and international borders. they pose substantial threats and serious repercussions to global public health security. while the asia-pacific region was generally regarded as the main epicenter of emerging infectious diseases, with outbreaks of avian flu, asian flu and severe acute respiratory syndrome (sars) [1] , the recent and unexpected emergence of zika pandemic spurred global concerns about pandemic preparedness capabilities particularly as it relates to training and deployment of healthcare workforce at a massive level, worldwide. despite coordinated global efforts, containing the "red alert" pandemic of zika remained a challenge, as both healthcare workers and public health advocates were uncertain about such disastrous contagion causing serious complications including congenital microcephaly in newborns and neurological deficits in adults [2, 3] . control measures were obtunded as public health advocates were initially speculative about the potential transmission route of zika, while clinicians in hospitals were irresolute, instituting multiple levels of care and management to tackle the complications of zika. this debacle gave rise to an urgent need to debate the circumstances under which the zika epidemic has challenged human intelligence behavior and capacity to battle the threat effectively and efficiently. as population explosions and uncontrolled human mobility across nations catalyzes rapid disease propagation, our next question is, what else above human intelligence could help resolve such unprecedented epidemic crisis? scientists believe that the time has come to institute analytic technologies-such as artificial intelligence (ai)-in healthcare to help prevent and resolve such large disease epidemics [4, 5] . adaptive ai applications could mould human behavior to practice preventive behaviors and disease control strategies [6] , thereby improving global health. this chapter will systematically discuss the dawn of ai technology in healthcare that could potentially empower the human population to tackle unprecedented infectious disease epidemics. the human population has witnessed four major revolutions till date (fig. 1) ; the foremost being the first industrial revolution that introduced steam engine to the world [7] . this was followed by the second industrial revolution that introduced electrical-energy based productions. the first information revolution was conceptualized during the third industrial revolution in the late 20th century. it was during this time that computers and internet-based knowledge began and has since then shaped human interactions. in early 21st century, the fourth industrial revolution accelerated the second information revolution. the entire phase of human daily functions transformed with the debut of ai, bringing together massive information flow from different specialties. these culminated in the rise of big data with systems integration across the internet of things (iot) and cloud computing systems. current revolutionary era is based on extreme automation for global connectivity, in which ai would definitely play an imperative role as a resource to utilize. at the peak of emergent multi-function contexts of ai and the rise of big data analytics, the united nations (un) in 2017 unified global experts to galvanize a dynamic consensus on the adoption and expansion of ai use in delivering good public care services [8] . succinctly, various stakeholders were assembled together in another un meeting to assess the role of ai towards achieving sustainable developmental goals (sdgs) [9] . from the healthcare perspective, massive data have been obtained from public health surveillance efforts with the advancement of ai. one major public health field that gained momentum to develop various ai applications for disease prevention was the infectious disease domain [5] . the human population is currently able to access potentially useful massive data sources of infectious disease spread through sentinel reporting systems, national surveillance systems (usually operated by national or regional disease centers such as the center for disease control (cdc)), genome databases, internet search queries (also called infodemiology and infoveillance studies) [10] [11] [12] , twitter data analysis [13, 14] , outbreak investigation reports, transportation dynamics [15] , vaccine reports [16] and human dynamics information [17] . with the influx of massive data volume, effective data integration, management and knowledge extraction systems are required [5] . epidemic modeling and disease-spread simulations form new horizons to understand the effects of citizen behaviors or government health policy measures [1] . a simple integrated effect of disease knowledge discovery is exhibited in fig. 2 . as humans, we are able to perform simple essential tasks such as object detection, visual interpretation and speech recognition. our interpretation is instantaneous when we look at an object or image, or when we hear voices or noises surrounding us. our next question is-could ai perform these essential intellectual tasks as well? the answer is absolutely yes, but in a different mode of function. while human interpretation is solely dependent on cognitive functions, ai requires mathematical algorithms to automate machines for execution of such functions [18] . machines here refer to programmable computers! an example is to visualize the cause of an outbreak; dengue, chikungunya or zika, of which these diseases are commonly caused by the vector mosquito. in massive epidemics, elimination of the vector is important, and human cognitive functions can never detect all mosquitos in an outbreak investigation area! however, this can be easily detected through deployment of ai in areas which have loads of mosquito vectors to facilitate control measures. figure 3 exhibits how human and ai technology interpret the vector differently. while human interpretation is instantaneous, ai evaluates the same image as humans do, but translated into codes [18] , facilitating massive detection. while ai aims to mimic human cognitive functions, it lacks intuitive behaviors. scientists postulate that such synthetic intelligence which could be on par with human intelligence can be called "computational intelligence." however, the primary goal [5] of ai was to create a system programming that is capable to think and act rationally like humans, although such machines may lack intuitive or emotional capabilities. as such, ai has been appropriately defined in simple and straightforward terms, as "a branch of computer science that deals with simulation of intelligent behaviors as humans using computers [19] ". in principle, there are three types of ai. if a machine is able to think as humans do and perform a task similar to human intellectual capabilities, then that machine functionality is referred to as artificial general intelligence [20] . if a machine performs a single task extremely well, this is known as artificial narrow intelligence [4] . if the same machine out-smart the best humans in all fields from scientific creativity to general wisdom or social skills, this is referred to as artificial super intelligence [4] . at present, virtually all contemporary ai application systems utilize artificial narrow intelligence. there are numerous concepts to function underlying ai applications in healthcare. based on the required functions, these concepts are clumped together to automate a single application-such as tracking infectious disease health seeking behavior. the following sub-sections summarize key concepts of different ai subsets adopted in emerging literature of infectious diseases. machine learning (ml). ml is a subfield of ai that implies learning from previous experiences (fig. 4) . the system finds solution to a problem by extracting fig. 3 interpreting the vector from the human and ai perspective. source da silva motta et al. [18] previous relevant data, learn from this data and predicts new outcomes [20] . ml applications are sub-divided into three categories: i. supervised learning: uses patterns of identified data (e.g. training data) ii. unsupervised learning: finds and learns from patterns of data (e.g. data-mining that involves identification of patterns in large datasets) iii. reinforcement learning: an extension of supervised learning that "rewards" and "punishes" when an application interacts with the environment. table 1 illustrates some common examples of supervised and unsupervised ml methods that are currently adopted and utilized to track health seeking behaviors during infectious disease epidemics [5] . deep learning (dl). dl is a specific subset of ml that uses neural networks (fig. 4) . in short, it is basically a synthetic replica of the human brain structure and functionality [4] . dl can execute multiple functions like image recognition and natural language processing (nlp). the system is capable of handling large datasets of information flow. image recognition. ai has the capability to process large amount of data about characteristics of a particular phenomenon in the form of images or signals [20] . motion images and sounds are examples of signals that could be analyzed using artificial neural networks (anns) [20] . recently, researchers from the usa proposed a system that could rapidly identify potential arbovirus outbreaks (mosquito, ticks or other arthropod borne viruses) [21] . the system identifies images of mosquito larvae captured and delivered by a group of citizen scientists. not only did the developed prototype facilitate collection of images, it also facilitated training of image classifiers for the recognition of a particular specimen. this sets a base for execution of expert validation process and data analytics. it was found that recognition of specimen in images provided by citizen scientists was useful to generate visualizations of susceptible geographical regions of arboviruses threat (fig. 5) . the system was capable of identifying mosquito larvae with great accuracy. the rapid identification of potential outbreak to a susceptible community could alert preventive behaviors and policy drafting in the quest to control potential epidemics. natural language processing (nlp). nlp bridges the gap between languages that humans and machines use to operate. algorithms are built to allow machines to identify keywords and phrases in an unstructured written text. ai applications then interpret the meaning of these texts for actionable knowledge [20] . expert systems (es). es incorporates expert-level competence to resolve a particular problem [20] . the system is constituted of two main components, namely knowledge base and a reasoning engine. it solves complex problems through reasoning a set of incomplete or uncertain information through a series of complex rules. in recent years, fuzzy logic, a set of mathematical principles for knowledge representation was crafted to accelerate the evolution of es. such strategy was utilized by a team of researchers from south africa to improve predictions of cholera outbreaks [22] . public reaction and behavior towards disease outbreaks could be difficult to predict. with the rise of big data analytics and a pool of ai applications in place, public health researchers were able to correlate population's behavior during an outbreak [5] . the following examples illustrate real life applications of ai during disease epidemics: twitter, a free social-networking micro-blogging service has enabled loads of users to send and read each other's "tweets (short, 140-character messages)." as important information and geo-political events are embedded within the twitter stream, researchers now postulate that twitter users' reactions may be useful for tracking and forecasting behavior during disease epidemics. the zika pandemic. most of the world's populations are living in endemic areas for common mosquito-borne diseases. the zika pandemic between the years of 2015 and 2016 marked the largest known outbreak, reaching a "red-alert" warning of multiple complications requiring global public health interventions. in such exigencies, population health behaviors are important for potential control measures. daughton and paul postulated that internet data has been effective to track human health seeking behaviors during disease outbreaks [12] . they used twitter data between 2015 and 2016 respectively to identify and describe self-disclosures of an individual's behavior change during disease spread. they combined keyword filtering and ml classifications to identify first-person reactions to zika. a total of 29,386 english tweets were analyzed. keywords include "travel," "travelling intentions," and "cancellations." individual demographic characteristics, users' networking and linguistic patterns were compared with controls. the study found variations between individual characteristics, users' social network attributes and language styles in 1567 twitter users. these users changed or considered to change their travel behaviors in response to zika. significant differences were observed between geographic areas in the usa, with higher discussion among women than men and some differences in levels of exposure to zika-related information. this finding concludes that applying ai concepts could contribute to better understanding on how public perceives and reacts towards the risks of infectious disease outbreak. the influenza a h1n1 pandemic. signorini and colleagues in 2011 analyzed twitter embedded data for tracking rapid evolvement of public concerns with respect to h1n1 or swine flu, while concurrently measuring actual disease activity [13] . the researchers explored public concerns by collecting tweets using pre-specified search terms related to h1n1 activity with additional keywords related to disease transmission, disease countermeasures and food consumption within the united states. they utilized influenza-like illness surveillance data and predicted an estimation model using supervised learning method in machine learning. the results showed that twitter was useful to measure public interest or concern about health-related events associated with h1n1. these include an observed periodical spikes related to user twitter activity that were linked to preventive measures (hand-hygiene practices and usage of masks), travel and food consumption behaviors, drug related tweets about specific anti-viral and vaccine uptake. they concluded that twitter accurately estimated influenza outbreak through ai applications [13] . the integration of internet data into public health informatics has been regarded as a powerful tool to explore real-time human health-seeking behaviors during disease epidemics. one such popular tool widely utilized is google trends, an open tool that provides traffic information regarding trends, patterns and variations of online interests using user-specified keywords and topics over time [11] . such adaptations formed two conceptualizations: the first was "infodemiology," defined as "the science of distribution and determinants of information in an electronic medium, specifically the internet, or in a population, with the ultimate aim to inform public health and public policy [23] ;" the second was "infoveillance," defined as "the longitudinal tracking of infodemiology metrics for surveillance and trend analysis [24] ." examining health-behavior patterns during dengue outbreaks. dengue is highly endemic across the south-east asian countries. recently, a group of researchers from the philippines conducted an infodemiology and infoveillance study by using spatio-temporal concepts to explore relationships of weekly google dengue trends (gdt) data from the internet and dengue incidence data from manila city between 2009 and 2014 [25] . they subsequently examined health-seeking behaviors using dengue-related search queries from the population. their findings suggested that weekly temporal gdt patterns were nearly similar to weekly dengue incidence reports. themes retrieved from dengue-related search queries include: "dengue," "symptoms and signs of dengue," "treatment and prevention of dengue," "mosquito," and "other diseases." most search queries were directed towards manifestations of dengue. the researchers concluded that gdt is a useful component to complement conventional disease surveillance methods. this concept could assists towards identifying dengue hotspots to facilitate appropriate and timely public health decisions and preventive strategies [25] . health-seeking behavior of ebola outbreak. an unprecedented ebola contagion that plagued most west african countries in 2014 marked the rise of global public health interest in pandemic preparedness interventions. millions of ebola-related internet hit searches were retrieved. with such high fluxes of health-seeking behavior using computers, a group of italian researchers' evaluated google trends search queries for terms related to "ebola" outbreak at the global level and across countries where primary cases of ebola were reported [26] . the researchers subsequently explored correlations between overall and weekly web hit searches of terms in relation to the total number and weekly new cases of ebola incidence. the highest search volumes that generated ebola related queries were captured across the west african countries, mainly affected by the ebola epidemic. web searches were concentrated across state capitals. however, in western countries, the distribution of web searches remained fixed across national territories. correlations between the total number of new weekly cases of ebola and the weekly google trends index varied from weak to moderate among the african countries afflicted by ebola. correlations between the total number of ebola cases registered in all countries and the google trends index was relatively high. the researchers concluded that google trends data strongly correlated with global epidemiological data. global agencies could utilize such information to correctly identify outbreaks, and craft appropriate actionable interventions for disease prevention urgently [26] . public reactions toward chikungunya outbreaks. the 2017 italian outbreak of chikungunya posed substantial public health concerns, catalyzing public interests in terms of internet searches and social media interactions. a group of researchers were determined to investigate chikungunya-related digital health-seeking behaviors, and subsequently explored probable associations between epidemiological data and internet traffic sources [27] . public reactions from italy toward chikungunya outbreaks were mined from google trends, google news, twitter traffics, wikipedia visits and edits, and pubmed articles to yield a structural equation model. the relationships between overall chikungunya cases, as well as autochthonous cases and tweet productions were mediated by chikungunya-related web searches. but in the allochthonous case model, tweet productions were not significantly mediated by epidemiological figures, instead, web searches posed significant mediating tweets. inconsistent associations were detected in mediation models involving wikipedia usage. the effects between news consumption and tweets production were suppressed in this regard. subsequently, inconsistent mediation effects were found between wikipedia usage and tweets production, with web searches as a mediator. after adjustment of internet penetration index, similar findings were retrieved with the adjusted model showing relationship between google news and twitter to be partially mediated by wikipedia usage. the link between wikipedia usage and pubmed/medline was fully mediated by google news, and differed from the unadjusted model. the researchers found significant public reactions to the chikungunya outbreak. they concluded that health authorities could be made aware immediately of such phenomenon with the aid of new technologies for collecting public concerns, disseminating awareness and avoiding misleading information [27] . expert systems are built upon the basis to act as a diagnostic tool to accelerate detection of infectious disease epidemics, determining the intensity or concentration of vector-agents within the triads of infectious disease dynamics. the malaria control strategy using expert systems. malaria constitutes a "red-alert" health threat to the african communities. a group of researchers from nigeria built an expert system for malaria environmental diagnosis with the aim of providing a decisional support tool for researchers and health policy-makers [28] . as prevailing malaria control measures were deemed insufficient, this group of researchers developed a prototype that constituted components of "knowledge," "applications," "system database," "user graphics interface," and "user components." the user component utilized java, while the application component used java expert system shell (jess) and the java ide of netbeans. the database component used sql server. the system was able to act as a diagnostic tool to determine the intensity of malarial parasites in designated geographical areas across africa. the proposed prototype proved useful and cost-effective in curbing malaria spread [28] . whereas ai is gaining increasing popularity and acceptance as a quick fix to the myriad of challenges faced with pandemic preparedness using traditional population-based approaches, it is not without its own limitations. even in resource-rich settings, there are challenges associated with building and updating the knowledge base of expert systems [29] , providing high-quality datasets upon which machine learning algorithms can be premised, and ethical issues associated with data ownership and management [20] . additionally, resource-limited settings are further plagued with constraints of poorly organized and integrated health systems, poor it and communication infrastructure, and socio-economic and cultural contexts [30, 31] that significantly impact successful implementation of ai systems. beyond these, the dynamics of human behavior and other environmental covariates (such as mass/social media, public emotions, public policy etc.) may not only influence the accuracy of epidemic disease modeling frameworks but also impact health seeking behavior during epidemics [32] . more than ever before, public health experts, it developers and other stakeholders must work together to address concerns related to scalability of ai for healthcare, data integration and interoperability, security, privacy and ethics of aggregated digital data. finally, the transparency of predictive ai algorithms have been called to question, particularly given their 'black box' nature which makes them prone to biases in settings of significant inequalities [33] . perhaps, it may be premature to describe ai as the future of healthcare given it is still in its infancy, however, it has become increasingly difficult to not acknowledge the substantial contributions of ai systems to the field of public health medicine. notwithstanding current challenges with the widespread adoption of ai particularly in resource-limited settings, the use of ai in providing in-depth knowledge on individuals' health, predicting population health risks and improving pandemic preparedness capabilities is likely to increase substantially in the near future [34] . further, the rapidly expanding mobile phone penetrance, developments in cloud computing, substantial investments in health informatics, electronic medical records (emrs) and mobile health (mhealth) applications, even in resource-constrained settings, holds significant promise for increasing use and scalability of ai applications in improving public health outcomes [35] . public health policy, practice and research will continue to benefit from the expanding framework of infodemiology and infoveillance in analyzing health information search, communication and publication behavior on the internet [23, 24] . advances in cryptographic technologies-including block chain is likely to allay fears and concerns with security, privacy and confidentiality of public digital data/information [36] . there is no doubt that ai is and will continue to revolutionize healthcare and population health. from prevention and health promotion to diagnosis and treatment, ai is increasingly being deployed to improve clinical decision-making, enhance personalized care and public health outcomes. in particular, ai offers enormous potential for cost-savings on therapeutic care given its predictive accuracy of potential outbreaks and epidemics and ability to enhance positive health seeking behaviors (at individual and population levels) during epidemics predicated upon robust infodemiology and infoveillance frameworks supported by expert systems, machine learning algorithms and mobile applications. amazing as the future of ai in healthcare seems, there are significant legal and ethical concerns that need to be addressed in order to pave way for robust implementation and scalability across a variety of socio-cultural, epidemiological, health system and political contexts. tracking infectious disease spread for global pandemic containment an update on zika virus infection neurologic complications associated with the zika virus in brazilian adults will artificial intelligence solve the human crisis in healthcare? bmc artificial intelligence for infectious disease big data analytics the human behavior-change project: harnessing the power of artificial intelligence and machine learning for evidence synthesis and interpretation recommendations for implementing the strategic initiative united nations: looking to future un to consider how artificial intelligence could help achieve economic growth and reduce inequalities detecting influenza epidemics using search engine query data google trends in infodemiology and infoveillance: methodology framework identifying protective health behaviors on twitter: observational study of travel advisories and zika virus the use of twitter to track levels of disease activity and public concern in the us during the influenza a h1n1 pandemic harnessing big data for communicable tropical and sub-tropical disorders: implications from a systematic review of the literature international air transport association (iata) vaccines meet big data: state-of-the-art and future prospects. from the classical 3is ("isolate-inactivate-inject") vaccinology 1.0 to vaccinology 3.0, vaccinomics and beyond: a historical overview social contacts and mixing patterns relevant to the spread of infectious diseases use of artificial intelligence on the control of vector-borne diseases, in vectors and vector-borne zoonotic diseases preterm birth associated with maternal fine particulate matter exposure: a global, regional and national assessment artificial intelligence (ai) and global health: how can ai contribute to health in resource-poor settings? image recognition of disease-carrying insects: a system for combating infectious diseases using image classification techniques and citizen science fuzzy expert systems and gis for cholera health risk prediction in southern infodemiology and infoveillance: framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the internet infodemiology and infoveillance: tracking online health information and cyber-behavior for public health using google trends to examine the spatio-temporal incidence and behavioral patterns of dengue disease: a case study in metropolitan manila assessing ebola-related web search behavior: insights and implications from an analytical study of google trends-based query volumes public reaction to chikungunya outbreaks in italy-insights from an extensive novel data streams-based structural equation modeling analysis building a computer-based expert system for malaria environmental diagnosis: an alternative malaria control strategy developing and using expert systems and neural networks in medicine: a review on benefits and challenges semantics derived automatically from language corpora contain human-like biases machine bias accounting for healthcare-seeking behaviors and testing practices in real-time influenza forecasts algorithmic transparency via quantitative input influence: theory and experiments with learning systems. security and privacy (sp) health intelligence: how artificial intelligence transforms population and personalized health tracking health seeking behavior during an ebola outbreak via mobile phones and sms adopting m-health in clinical practice: a boon or a bane? acknowledgements we thank the ministry of health malaysia for the support to publish this chapter. key: cord-193136-7g6qr73e authors: bhattacharya, sujit; singh, shubham title: visible insights of the invisible pandemic: a scientometric, altmetric and topic trend analysis date: 2020-04-22 journal: nan doi: nan sha: doc_id: 193136 cord_uid: 7g6qr73e the recent sars-cov-2 virus outbreak has created an unprecedented global health crisis! the disease is showing alarming trends with the number of people getting infected with this disease, new cases and death rate are all highlighting the need to control this disease at the earliest. the strategy now for the governments around the globe is how to limit the spread of the virus until the research community develops treatment/drug or vaccination against the virus. the outbreak of this disease has unsurprisingly led to huge volume of research within a short period of time surrounding this disease. it has also led to aggressive social media activity on twitter, facebook, dedicated blogs, news reports and other online sites actively involved in discussing about the various aspects of and related to this disease. it becomes a useful and challenging exercise to draw from this huge volume of research, the key papers that form the research front, its influence in the research community, and other important research insights. similarly, it becomes important to discern the key issues that influence the society concerning this disease. the paper is motivated by this. it attempts to distinguish which are the most influential papers, the key knowledge base and major topics surrounding the research covered by covid-19. further it attempts to capture the society's perception by discerning key topics that are trending online. the study concludes by highlighting the implications of this study. coronaviruses are viruses that circulate among animals and are named because of the crownlike spikes (protein spikes) that protrude from their surface resembling the sun's corona. the first transmission of this type of virus from animals to humans happened in 2002 in the guangdong province of china which resulted in sars (severe acute respiratory syndrome). bats were thought to be the potential source of this virus. a novel coronavirus (ncov) was identified in early january 2020 which was traced to the severe pneumonic outbreak of an undocumented cause in early december 2019 in the city of wuhan, china. similar to sars, bats are seen as the potential source from which this virus has spread to humans. due to rapid spread of the virus within a short time globally resulting in health emergencies in a number of countries, who declared this as a pandemic on 11 th march 2020. initially the virus was named 2019-ncov but later was named sars-cov-2 due to its "genetic relationship to the sars-cov-1 virus"(medscape, 2020) the entire human population is at potential risk as being a new virus nobody has prior immunity to it. there is no vaccine and no specific treatment for the disease and is highly transmissible. epidemiological estimation at present is that on an average, one infected person will infect between two to three other people. the spread has been from respiratory droplets and from contaminated surfaces. the world is facing a common challenge; how to control the spread of and what can be the effective interventions to control mortality. the early examples from china suggested the most effective method to control the spread of the virus is through lockdowns and social distancing measures. the example of south korea suggested testing as the major component of the mitigation measures. some studies pointed out the importance of hand hygiene and face masks in controlling the virus. with new hotspots emerging, the number of new cases and those not able to recover are raising new concerns every day. risk and uncertainty behind this disease control has generated a global concern for health, economy, and for persons at large. the alarming spread of the virus has shocked people across the world pushing among others researchers to understand the virus-its structure, transmission, replication mechanism, latency, etc. and promising interventions that can effectively control it. extensive global efforts are undertaken to develop vaccine and drug. this is unsurprisingly leading to huge volume of research activity within a short period of time increasing at an exponential rate. as the recent editorial published in the lancet highlights "the whole-genome sequence of sars-cov-2 had been obtained and shared widely by mid-january, a feat not possible at such speed in previous infectious disease outbreaks" the editorial points out the importance of the need for development of effective diagnostics, therapeutics and vaccine for the virus. examining from the dimensions database as of 22 april, it was found that 1633 clinical trials are being conducted on the virus and 171 policy documents have been published so far. the number of research papers, clinical trials at different phases within such a short period is unprecedented and shows the intensive efforts of the global research community to understand the different aspects of this disease and address it. seven patents have also been granted. it is important to capture insights of influential research and innovation from this ongoing activity for policy makers and research scholars from cross-disciplinary areas to build up further on this valuable repository. societal impact and what aspects are of concern to the people at large are difficult to capture. one useful method would be from online trends surrounding this disease that would indicate to some extent the key issues that are influencing the society at large. the present study is motivated by this and applies tools and techniques of scientometrics to uncover insights from research papers. scientometrics applies various mathematical and statistical techniques to capture insights of research activity from research papers and patents and other published sources including online sources (altmetrics) by constructing various types of indicators. citation based analysis is a prominent method to capture academically significant and theoretically relevant material (see for example glanzel, 2003) . keeping in view the research activity in this area started primarily with the outbreak of this disease, impact captured through citations would not give a correct picture as citations takes time to accrue. this is true for research paper as well as patent citations. citations that influence current research activity would however be useful to construct the present knowledge base. one of the useful method that can do so is based on cocitation analysis which captures frequency with which two documents are cited together (small, 1973) . co-citation establishes an intellectual relationship with earlier literature in a field/subfield/area of research; strength of relationship based on frequency of co-citation pairs. the rise of social networking websites like twitter, facebook etc. provides researchers a wider scope to share their scholarly publications. altmetrics allows to track and capture online impact of scholarly research and thus broadly indicates papers that are influencing the research community. to put it in a proper perspective, one can borrow from william (2017), "altmetrics are measurements of how people interact with a given scholarly work". altmetrics or article level metrics according to das and mishra (2014) is a "new trendsetter" to measure "impact of scientific publication and their social outreach to intended audience". it reflects "a scholarly article's popularity, usage, acceptance and availability" by using an altmetric score. google trends which was launched in 2006, primarily shows how frequently a particular search term is entered in comparison with all other search terms in different regions and languages (google, 2017). in google trends level of interest in a topic is approximated using search volume of google. sullivan (2016) estimated searches on google trends reached 2 trillion in 2016! thus, this is one of the most significant source of data if it is properly analysed. one of the most influential study was by ginsberg et al. (2006) which showed that google trends traced and predicted the spread of influenza earlier than the centers for disease control and prevention. jun et al. (2018) provides a good assessment of research studies in the past decade which have utilized google trends. they highlight the diverse fields in which this has been used for, from merely describing and diagnosing research trends to forecasting changes. according to mavragani et al. (2018) "google trends shows the changes in online interest for time series in any selected term in any country or region over a selected time period, for example, a specific year, several years, 3 weeks, 4 months, 30 days, 7 days, 4 hours, 1 hour, or a specified time-frame." they argue that as the internet penetration is increasing web based search activity has become a valid indicator of public behaviour. the paper positions itself in this direction; applying various tools and techniques of scientometrics, altmetrics and google trends to draw meaning from the huge volume of research papers and online activity surrounding this pandemic. the study attempts to answer the following research questions:  what are the key papers that captures the most relevant research, areas and topics on covid-19?  what is the knowledge base that influences current research on this pandemic?  what are the key aspects of this pandemic that is influencing the society at large? the study has used various types of data sets and analytical techniques as highlighted below to capture the research trends and also assess this disease influence on the society. the dimensions database (www.dimensions.ai) was used for this study. this database has various unique features which makes it very useful to capture various aspects of research activity. it provides dynamic altmetrics score for each article. the database unlike source based classification (journal classification) used in indexing articles in sci and scopus database uses article level classification. only when an article cannot be classified individually due to lack of information, it uses the fields of research (for) classification system. the for 2 has three hierarchical levels: divisions (represents a broad subject area or research discipline), with the next two levels groups and fields representing increasingly detailed subsets of these categories. in for there are 22 divisions, 157 groups and 1238 fields. dimensions has incorporated only the groups in its classification system. thus classification article level provides a more informed assessment of the topic covered by it then based on journal level classification which is a macro level classification. these features motivated us to use this database for this study. the articles on this virus were extracted using the search string "covid-19" or "sars-cov-2" or "sars-cov2" or "2019-ncov" on april 12, 2020 from this database. the final search string was developed based on review of contemporary studies and deleting those search keywords that lead to noises. for example, it was found that ncov which some studies have used also identifies papers that cover mers (middle-east respiratory syndrome). this was first reported in 2012, was initially called novel coronavirus or ncov as it was a species of coronavirus. many studies had applied search string without hyphen which also results in extracting papers not covering this disease. the search string applied on the publications database of dimensions resulted in 9146 papers, containing 7332 articles and 1814 pre-prints. this data set of 9146 papers were further used for analysis. influential papers were distinguished by using altmetric score which is a weighted count of all the online attention of a research paper. the altmetrics data was captured from dimensions database which draws data from altmetrics.com of capturing online activity of research papers on facebook, twitter, blogpost, news reports etc. the score changes as people mentioning the paper increases (only one mention per user is considered). each category of mention carries different base amount so a news article contributes more than a blog post which in turn contributes more than a tweet in the final score. country wise analysis showed that around 78 percent of the total papers were contributed by ten countries. further analysis of research activity of the ten identified countries was done using altmetric and citation analysis. word cloud provides a high visual representation of concepts that a paper had frequently applied. it is based on burst algorithm that captures the sudden rise in the usage of a word. mane and borner (2004) highlighted the usefulness of burst words as according to them "it helps humans mentally organize and electronically access and manage large complex information spaces". using r programming tools, word cloud was constructed from keywords of the data set. the words with higher frequency in the overall corpus of papers (herein 9146) have a larger font size and acquires more space in a visualisation. word cloud was used to get visualisation of 70 most frequent words; the number of words chosen was limited by the clarity of visualisation. co-citation analysis helps to capture papers that are co-cited together in a large number of papers. the highly co-cited papers is seen as the core knowledge base of research area at a particular period. this analysis was undertaken to identify the key knowledge base behind the identified papers. dimensions database was used to extract a bibliographic mapping file for the papers. two software's pajek and vosviewer were used for co-citation analysis. initially the bibliographic mapping file was run on the vosviewer software to identify the most co-cited papers. the co-cited papers were identified at four levels (trim levels) to have a deeper insight of the core knowledge base: level 1 identified 51 papers co-cited 77 or more times; level 2 identified 26 papers co-cited 126 or more times; level 3 identified 10 papers co-cited 277 or more times, and level 4 identified 5 papers co-cited 463 or more times. for each of the trim levels a network file was obtained from the vosviewer software. the network file was then run on pajek software to create a refined co-citation network map so as to avoid overlapping of nodes. the final visualisation was done for the refined map in the vosviewer software. data for the policy documents referencing these top ten co-cited papers was done by accessing altmetrics.com directly from the dimensions database. another question which the study explored is the impact of this virus on the society primarily what are the key aspects of and related to this disease that has influenced the society at large. google trend analysis of key topics have been undertaken to capture this aspect. google trends website (https://trends.google.com/trends/?geo=us) was first accessed on 10 th april 2020. the topics were chosen based on closely monitoring the news items, and also finally choosing from a large set of topics. choice for example of 'pandemic' was seen to have initial burst but declined quickly. vaccine was trending highly but we found lot of noise in this term. the final six topics chosen were "social distancing", "quarantine", "covid-19", "coronavirus", "face mask" and "hydroxychloroquine". data for each of the topics was finally taken on 16 april, 2020. for country specific comparison data for five countries having maximum cases of covid-19 namely usa, italy, spain, france, germany and two emerging economies india and brazil was also obtained. hydroxychloroquine was not used in country specific search as it was only visible trending for three countries among the chosen seven countries. google trend analysis was not done for china as there is much restricted access to google in that country. one of the first important observation is the intensity with which research on covid-19 and related aspects is going on globally. search conducted in two different time periods, 28 th march and then on 12 th april showed that 2172 and 9146 papers were published in these two periods; almost 320 percent growth during such a short time. the insights that we draw from our analysis of the 9146 papers is presented in different sections below ten most popular research papers among the 9146 covid-19 papers were extracted on the basis of their altmertrics score on april 12, 2020. table 1 highlights these influential papers. 2020), the study most popular on social media platforms (number of tweets more than three times the next popular paper) commented that "sars-cov-2" is the seventh coronavirus to infect humans". the study found that sars-cov-2 is not a product of purposeful manipulation and is most likely the result of natural selection of human or human-like ace2 receptor. the study also found that sars-cov-2 spike protein has high affinity to bind to human ace2 receptor. the study also estimated that the undocumented cases contagiousness or transmission rate was 55% of documented infections, yet 79% of documented infection cases were due to these undocumented infections. the suggestion of this study that undocumented infections "isolation and identification is necessary to fully control the virus" is very important and the spread of this virus may be seen as a consequence of this. this study also was cited in policy documents. leung et al. (2020) explored "the importance of respiratory droplet and aerosol route of transmission" by quantifying the "amount of respiratory virus in exhaled breath of participants" that have acute respiratory virus illness (ari). the 246 participants were divided in two groups, one wearing surgical face mask and other not wearing face mask. the study found that surgical face masks can efficaciously reduce the respiratory droplet emission of influenza virus particles but not in aerosols. they also found that surgical face masks can be used by ill patients of covid-19 to reduce "onward transmission". face mask is getting increasing attention and now being incorporated as essential guideline in health policies of different countries. table 2 points to some interesting aspects of research activity in this area. these ten countries account for almost 78 percent of total papers with china and usa accounting for 45 percent of the total. china, usa and uk are actively collaborating among themselves and also with other countries. this is a good indication as global collaborative efforts, pooling each other resources are required to meet the challenges posed by this disease. a few leading universities can be discerned which are actively involved in this research. popularity of a paper can also be seen influenced by journals; papers with high altmetrics score strongly correlate with journals that have high reputation in the field (high impact factor, leading journal of the community). table 3 highlights the areas covered in the covid-19 papers. the table provides a broad indication of intensity of research happening in different fields. figure 1 presents a word cloud of most frequently used terms in covid-19 papers. the word cloud shows key aspects that have been part of many studies. the word cloud maps the topics of research surrounding this disease. the two keywords, for example "pandemics" and "china" that have maximum occurrence in papers indicated by large font size which shows that these two aspects were discussed in many papers. it is known that china was the source of this infection and who declared this disease as a pandemics. thus increasing research mention of these two keywords is not surprising. coronavirus primarily affects animals, sars disease as a result of transmission of coronavirus from animal to humans, travel has contributed maximum to the spread of this disease, are all visible prominently in this word cloud. thus, examination of the word cloud is useful to have a broad view of key areas of research in the 9146 papers. figure 2 shows co-citation networks at four trim levels. it can be observed from figure 2 that trim level 4 that contains top 5 co-cited papers is a complete cluster. a complete cluster according to gmür (2003) is when "each reference is connected to other references and there is no dominant document within the cluster". table 4 highlights the details of these co-cited papers at trim level 3 which identifies top 10 cocited papers. it also includes the top 5 co-cited paper at trim level 4 (refer methodology for details). pandemic's influence on the society figure 3 provides global google trends of six topics (refer methodology for details) currently talked about extensively on social media, news reports or in general public discussion. it can be observed the otherwise flat line of covid-19 started seeing spikes in late february, 2020. this is because the term came into existence when who on feb 11 named the disease from the virus as covid-19 (coronavirus disease 2019). it can also be observed that the term reached maximum level of interest during the end of march as cases started showing significant increase in countries usa, india etc. measures like quarantine has strong societal influence and thus useful to look at trend in this topic. another topic "pandemic" saw maximum interest around march 11, as who announced covid-19 pandemic on that day. it can also be seen that the interest about this topic fell shortly thereafter. as discussed in the methodology, this topic hence was not chosen further in this study for google trend examination. hydroxychloroquine term has seen a great amount of interest from middle of march, 2020. this can be traced to the study by reputed french physician and microbiologist didier raoult who highlighted the use of this antimalarial drug in the treatment of this disease. it led to french president and us president endorsing this line of treatment which created favourable public opinion in many countries towards this drug. liu et al. (2020) whose work has also attracted high altmetrics attention found the drug to be effective in "inhibiting sars-cov-2 in vitro". this line of treatment and the robustness of the methodology and findings have also generated critical comments, see for example grens (2020) . india has become a key source for this medicine and already exported it to number of countries. thus a high degree of activity in google as seen through google trend can be due to various factors, positive as well as adverse reactions. similar public opinion generated by studies and endorsement can be seen in youtube searches in face mask. source: google trends figure 4 provides the comparison of google trends of "social distancing", "covid-19", "quarantine", and "lockdown" and "face mask" from 16 feb to 14 april 2020. figure 4 (a) presents a global picture of these topics and figure 4 (b) shows data for five countries having maximum cases of covid-19 namely usa, italy, spain, france, germany and two emerging economies india and brazil. three of the five topics chosen restrict people's movement. social distancing which basically means keeping a safe distance of around 6 feet from others and avoiding places where this kind of distance cannot be made like schools, workplaces, a sports game or a temple. the second one is quarantine which applies to a person who have been in exposed to coronavirus or patients having coronavirus. the person has to avoid contact with people till the specified incubation period of the virus to see if they develop symptoms. third lockdown, the term mainly used to describe the confinement of prisoners to their cells has now a changed definition during this outbreak. through lockdown people are not allowed to leave their local area, building and it is used as a control measure to prevent covid-19 disease transmission. it can be seen from figure 4 (b) that "lockdown" is the most popular topic worldwide as well as at the country level. according to world economic forum 2.6 billion people i.e. one thirds of the world population are under some kind of lockdown. 3 this figure alone constitutes india's 1.3 billion, the largest lockdown in the world. thus it is unsurprising to see the popularity of this topic over others in india. these control measure have major economic, psychological and social impacts and in turn affects lives of all the people involved. a worrying trend is the low comparative interest in social distancing in most of the countries and almost negligible comparative interest in countries like indian and brazil. the stand taken by brazil through her president of opening up and has been critical of measures like social distancing and lockdown may have contributed to this type of trend. proximal origins of sars-cov-2 a trial of lopinavir-ritonavir in adults hospitalized with severe covid-19 challenges of coronavirus disease 2019 epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in wuhan, china: a descriptive study genesis of altmetrics or article-level metrics for measuring efficacy of scholarly communications: current perspectives the cognitive paradigm : cognitive science, a newly explored approach to the study of cognition applied in the psychology of scientific knowledge and of education in science . rug. faculteit psychologische en pedagogische wetenschappen hydroxychloroquine and azithromycin as a treatment of covid-19: results of an openlabel non-randomized clinical trial detecting influenza epidemics using search engine query data co-citation analysis and the search for invisible colleges: a methodological evaluation bibliometrics as a research field: a course on theory and application of bibliometric indicators journal publisher concerned over hydroxychloroquine study clinical features of patients infected with 2019 novel coronavirus in wuhan ten years of research change using google trends: from the perspective of big data utilizations and applications hydroxychloroquine, a less toxic derivative of chloroquine, is effective in inhibiting sars-cov-2 infection in vitro mapping topics and topic bursts in pnas google trends in infodemiology and infoveillance: methodology framework analysis of the capacity of google trends to measure interest in conservation topics and the role of online news coronavirus disease 2019 (covid-19): a global crisis co-citation in the scientific literature: a new measure of the relationship between two documents google now handles at least 2 trillion searches per year the donut and altmetric attention score: an at-a-glance indicator of the volume and type of attention a research output has received document co-citation analysis to enhance transdisciplinary research altmetrics: an overview and evaluation characteristics of and important lessons from the coronavirus disease 2019 (covid-19) outbreak in china the authors thank dr vivek singh, professor, department of computer science, banaras hindu university for providing access to dimensions database which helped in framing the pilot study for this research. we are grateful to dimensions for providing us direct access. glad to see the initiates they have taken in promoting covid-19 research. note: comparative interest over time of social distancing for india and face mask and social distancing for brazil were negligible, so these terms have been removed for the graph for these countries. covid-19 disease took the world by surprise. the alarming spread of the disease, challenge to control its spread and its grave health consequences, lack of vaccine or effective drug among others has prompted researchers to actively work on various aspects of this disease. this has led to a huge volume of research output within a short period of time. the various aspects surrounding this disease has also effected society's perception of this disease. drawing insights from this huge volume of research output is a challenge as well as an exercise of significant importance for the policy makers and research community. scientometrics provides various tools and techniques to uncover insights from research papers. altmetrics and google trends are novel approaches to track online impact. these tools and techniques was used in this study to analyse 9146 papers that were discerned from the dimensions database covering covid-19 for the period upto 12th april, 2020. content analysis of a set of ten key papers was also undertaken to draw qualitative insights of these influential papers. some of the insights from this study reveals the key areas in which research has progressed. their visible impact can be seen in their altmetrics score and more directed impact in their citation in key policy documents. key policy influence such as period of quarantine, treatment type, use of face masks, population more vulnerable to disease etc can be traced to the influential papers that were discerned from this study. it would be however fallacy of generalisation if we say that these papers were the only influential factors behind the policy decisions. it is also interesting to see how papers had attracted attention from different online sources like twitter, news, blog, and facebook. thus, the importance of these sources in influencing research impact calls for researchers to aggressively use these modes for dissemination of their study findings. another important insight comes from the active collaboration seen in research papers. china and usa drive this research globally and are also actively engaging with other countries in research. this finding is drawn from top ten active countries which constitute almost 78% of the total research output as visible from research papers. word cloud showed the influential topics of research surrounding covid-19 research. this type of visual maps provide a good indication of key topics that constitute research activity in a area at a period of time. co-citation analysis identified the knowledge base that influences current research on this pandemic. it was observed that all of the top co-cited papers were published in high impact factor journals. it was also observed that most of the studies were driven by epidemiology and clinical characteristics of the disease. google trends analysis showed how the disease shapes the public opinion on certain topics. global trending topics incorporated interest over time in web, news, google shopping and youtube searches. a sudden rise in a topic's interest could be traced to various exogenous factors. the trend, for example in hydroxychloroquine, use of this antimalarial drug in the treatment of this disease could be seen the interplay of various forces, the impact of research paper, political endorsement and critical questioning of the research community, doctors etc to the effectiveness of this line of treatment. the trends observed in measures like lockdown, social distancing and quarantine at global and country level showed the societal increasing concern with these aspects.the findings of this study suggests how the research and public interest has been shaped around this disease. with so much information surrounding this disease, the study provides a space for understanding its various aspects. the study however is limited as it has not examined patents, clinical studies and policy documents. this may provide indications of implementable aspects that draw from research. future research intends to examine this. key: cord-256094-f85xc5uu authors: milinovich, gabriel j; avril, simon m r; clements, archie c a; brownstein, john s; tong, shilu; hu, wenbiao title: using internet search queries for infectious disease surveillance: screening diseases for suitability date: 2014-12-31 journal: bmc infect dis doi: 10.1186/s12879-014-0690-1 sha: doc_id: 256094 cord_uid: f85xc5uu background: internet-based surveillance systems provide a novel approach to monitoring infectious diseases. surveillance systems built on internet data are economically, logistically and epidemiologically appealing and have shown significant promise. the potential for these systems has increased with increased internet availability and shifts in health-related information seeking behaviour. this approach to monitoring infectious diseases has, however, only been applied to single or small groups of select diseases. this study aims to systematically investigate the potential for developing surveillance and early warning systems using internet search data, for a wide range of infectious diseases. methods: official notifications for 64 infectious diseases in australia were downloaded and correlated with frequencies for 164 internet search terms for the period 2009–13 using spearman’s rank correlations. time series cross correlations were performed to assess the potential for search terms to be used in construction of early warning systems. results: notifications for 17 infectious diseases (26.6%) were found to be significantly correlated with a selected search term. the use of internet metrics as a means of surveillance has not previously been described for 12 (70.6%) of these diseases. the majority of diseases identified were vaccine-preventable, vector-borne or sexually transmissible; cross correlations, however, indicated that vector-borne and vaccine preventable diseases are best suited for development of early warning systems. conclusions: the findings of this study suggest that internet-based surveillance systems have broader applicability to monitoring infectious diseases than has previously been recognised. furthermore, internet-based surveillance systems have a potential role in forecasting emerging infectious disease events, especially for vaccine-preventable and vector-borne diseases. electronic supplementary material: the online version of this article (doi:10.1186/s12879-014-0690-1) contains supplementary material, which is available to authorized users. prudent detection is a cornerstone in the control and prevention of infectious diseases. traditional infectious disease surveillance systems are typically characterised by a bottom-up process of data collection and information flow; these systems require a patient to recognise illness and seek treatment and a physician or laboratory to diagnose the infection and notify the relevant authority [1, 2] . for emerging infectious disease events, this process is reported to take, on average, 15 days from onset to detection and a further 12-24 hours for the world health organization to be notified [3] . the development and implementation of more efficient systems for gathering intelligence on infectious diseases has the potential to reduce the impact of disease events. internet-based surveillance systems are one such system [4] . internet-based surveillance systems produce estimates of disease incidence through analysis of various digital data-sources. targeted sources include internet-search metrics, online news stories, social network data and blog/ microblog data [4] . currently, the most promising approach appears to be those based upon monitoring of internet search behaviour. this approach works on the premise that people will actively seek information on diseases they develop and that estimates of disease activity with the community may be developed by monitoring the frequency of related internet searches. through targeting people earlier in the disease process, internet-based systems are able to access a larger fraction of the community and produce more timely information. furthermore, internet-based surveillance systems are intuitive and adaptable, cheap to run and maintain (once established), do not require a formal public health network and have the capacity to be automated and operate in near-real time. despite these advantages, internet-based surveillance systems have a number of significant shortcomings and must not be considered an alternative to traditional surveillance approaches [5] . firstly, as these systems crowd-source data, resolution will be contingent on the size of the population serviced and may be further limited by national communications infrastructure availability and distribution [6] . secondly, as internetbased surveillance systems are limited to people who use the internet to source health information, there is the potential that estimates produced by these systems may not accurately reflect the entire community [7] . finally, as internet-based surveillance systems essentially rely upon self-reporting, bias may be introduced through differences in internet usage between sectors of the community (the elderly, for example, may not use the internet as a source of health information, despite being a high-risk group for many infectious diseases) and/or through media driven interest in emerging disease events [4] . infectious diseases surveillance systems have been developed using internet search metrics to estimate incidence of influenza (google flu trends) [8] and dengue (google dengue trends) [9] . currently, operational systems that utilise this approach are limited, however, studies of the potential for internet-based surveillance have been conducted for a range of other infectious diseases, including: acute respiratory illness [7] , aids [10] , chickenpox [11, 12] , cryptosporidiosis [13] , dysentery [10] , gastroenteritis [11] , hepatitis [14] , listeriosis [15] , lyme disease [16] , methicillin-resistant staphylococcus aureus [17] , norovirus [18] , respiratory syncytial virus [6] , rotavirus [19] , scarlet fever (streptococcus pyogenes) [10, 20] , salmonella [21] , tuberculosis [10, 22] and west nile virus [6] . previous studies have focused on single diseases, or a small number of diseases, and the justification of the focus on a particular disease has been specific to each study. the published results have largely been promising; however, to date there has been no systematic, generalizable analysis to identifying diseases that are suited to monitoring through the analysis of internet-search metrics. the underpinning goal of this study was to provide direction for future approaches to developing digital surveillance systems; such as the development of predictive models and/or integrative surveillance models that draw upon multiple traditional and digital data source to create estimates of disease within the community. this study, however, did not aim to develop actionable surveillance systems, produce predictive models of infectious disease based on internet-based data or to identify the best search terms for use in these models. rather, this study aimed to determine which diseases have most promise for monitoring by surveillance systems built on internet search metrics; this was achieved by assessing the level of correlation between a wide range of infectious diseases and internet search term metrics. finally, this study aims to identify diseases for which internet-based data could be used to create early warning systems. surveillance data on notifiable infectious diseases were collected from the national notifiable disease surveillance system (nndss) which is maintained by the australia government department of health (doh) [23] . monthly notifications (case numbers) aggregated at state/territory and national level, were downloaded for the period of january 2004 to september 2013. a full list of notifiable diseases in australia and case definitions can be accessed through the doh webpage [24] . sixty-four diseases are monitored and these are categorised in the nndss as belonging to one of eight groups: bloodborne diseases; gastrointestinal diseases; other bacterial diseases; quarantinable diseases; sexually transmissible infections; vector-borne diseases; vaccine preventable diseases; and zoonoses. for the purpose of consistency, we have reported diseases according to these groupings. whilst notifiable, data were not downloaded for human immunodeficiency virus infection/acquired immunodeficiency syndrome, creutzfeldt-jakob disease or variant creutzfeldt-jakob disease because surveillance for these diseases is not performed by doh or for severe acute respiratory syndrome, because reporting to the doh is informal; as such, these diseases are not listed on the nndss. in the construction of google flu trends model, the authors identified search terms by performing correlations between influenza-like illness data from the us cdc and the top 50 million google search queries performed in the us over the corresponding period [8] . such data is not available to the public and an alternative approach to identification of search terms was required; two approaches were used. firstly terms related to diseases, the aetiological agents and colloquialisms (such as "hep" for hepatitis or "flu" for influenza) were manually identified. secondly, google correlate (www.google.com/trends/correlate) was queried using monthly surveillance data (described above). google correlate provides a list of up to 100 search terms that correlate most highly with the query data. to account for potential language shifts that may have affected search behaviour [4] , this was performed three times using surveillance data covering the periods 2004-13, 2007-13 and 2011-13. up to 300 search terms were downloaded from google correlate for each notifiable disease (100 search terms per period analysed) and manually sorted; any term related to the queried notifiable disease was included, regardless of the nature of the potential association suitable terms were combined with the manually identified search terms to create a list of search terms (see additional file 1). no attempt was made to filter search terms based upon biological plausibility; any term that may be perceived to have any association with the disease of interest was included. search frequencies for terms of interest were collected through google trends (www.google.com/trends/). all data extractions were performed on the 22nd of october, 2013. google trends was queried using each of the identified terms at a national and state/territory level using the entire time range available (2004-present). google trends presents search frequency as a normalised data series with values ranging from 0 to 100 (with 100 representing the point with the highest search frequency and other points scaled accordingly); functionality for exporting search frequency data as a .csv file is provided. for the purpose of privacy, data are aggregated at a daily, weekly or monthly level (or are restricted if there is insufficient search volume). the level of aggregation applied is determined by the period analysed and the search frequency; the level of aggregation is not able to be specified by the user. as the notifiable disease surveillance data used was in monthly format, monthly indices of query search frequencies were required. monthly indices are displayed graphically by google trends when querying periods greater than 36 months; rather than downloading. csv files, a script was developed to scrape data from the google trends webpage, allowing the problems associated with the level of data aggregation to be overcome. analyses were performed at both national and state levels for the period 2009-13. as state-level search frequency data were not always available, particularly for less common diseases (due to low search frequency at this level of disaggregation), correlations between state-level notification data and national search frequency data were also performed. owing to the large number of correlations performed in this study, bonferroni adjustments [25] were applied to significance levels by the equation 1-(1-α) 1/n ; all p-values reported in this document correspond to onetailed tests. spearman's rank correlation coefficients were used to rank performance. time-series cross correlations were performed to assess linear associations between disease notifications and google trend search indices. cross correlations were calculated using lag values for google trends data ranging from −7 to 7. this range allowed for assessment of biologically plausible associations that were relevant to the development of early warning systems. cross correlations were performed on national data using ibm spss version 21 (spss inc; chicago, il, usa). seasonal differencing was applied (value 1) to all analyses to remove cyclic trends. whilst all available data (2004-13) were downloaded, analyses for this study were focused on the most recent five years (2009-13) as preliminary data analyses indicated that google trends data were not available prior to 2009 for numerous search terms ( figure 1 ; panels 2, 4, 9, 12, 16 and 17). additionally, shifts in language are known to affect surveillance systems built upon textual data [4] . the shortened period (2009-13) was selected to minimise the effects of language shifts. however, this period still provides the requisite 50 pairs of observations for performing cross correlations [26] . in this section we discuss analyses of time series data. briefly, the time series analysed were monthly case numbers for the 64 infectious diseases monitored by the australian government's national notifiable disease surveillance system (nndss) and google trends monthly search metrics for related internet search terms. in total, search 164 terms were analysed in this study; this ranged from a single term for some diseases, up to 14 search terms for influenza and 35 search terms for pneumococcal disease. the majority of terms could be categorised as diseases or aetiological agents ("brucellosis" or "brucella"), colloquialisms ("flu", "hep" or "tb"), symptoms ("cough", "white discharge" or "cervical mucus") or medication or general health/treatment related queries ("whooping cough treatment", "symptoms of dengue" or "flu and pregnancy"). a few terms that may have environmental ("flash floods" for leptospirosis) or behavioural ("african tours" for malaria) meanings were also included. a full list of the search terms analysed is presented in the supplementary material. evaluation of the bivariate associations between surveillance and corresponding search frequency data was performed using the spearman's rank correlation. spearman's rank correlations for the 18 top ranked notifiable diseases and terms are presented in figure 2 and raw data for the corresponding diseases and search terms are presented in figure 1 . results of spearman's correlations indicated 17 diseases to be significantly correlated (p < 0.05; bonferroni corrected: p < 2.43e −04 ) with at least one search term; p-values for 12 of these were <0.0001 (bonferroni corrected: p < 4.74e −07 ). marked differences were observed in correlations between the various disease groups. correlations for vaccine-preventable diseases were generally highest with six of fourteen exhibiting strong (rho =0.60-0.799) or very strong (rho =0.80-1.00) correlations, followed by sexually transmitted infections (2/6), the vector-borne diseases (3/9), blood-borne diseases (1/6), other diseases (1/4), zoonoses (0/8), gastrointestinal infections (0/11) and, finally, quarantinable diseases (0/6). state level correlations are also reported in figure 2 . consistency between state correlations were variable with some diseases exhibiting reasonable consistency (pertussis; rank 8), whilst others were inconsistent (hepatitis c; rank 11). results of cross correlations are demonstrated in figure 3 . cross correlation results should be interpreted as product-moment correlations between the two time series; they allow dependence between two time series to be identified over a series of temporal offsets, referred to as lags. lag values indicate the degree and direction of associations. a lag value of −1 indicates that correlations were performed using time series data for which the first series (google trends' data) has been shifted backwards one unit (a month). conversely, a lag value of 1 indicates that the primary series had been shifted forward one unit. significant positive correlations for lag vales of ≥1 or above are of most interest in the context of this study as they indicate a positive relationship between the two time series with google trends data leading the notifications (a pre-requisite for google trends data to be a suitable early warning tool). it should also be noted that seasonal differencing was applied to cross correlations to remove cyclic seasonal trends. disease notifications positively correlated at a lag of one month (lag 1) with search term frequency for 12 of the 17 diseases that exhibited significant spearman's rank correlations. overall, 15 of the 64 notifiable diseases exhibited significant, positive correlations at lag of one month. significant positive associations were observed for four of the nine vector-borne diseases (barmah forest virus infection, dengue virus infection, murray valley encephalitis virus infection and ross river virus infection), six of the 14 vaccine preventable diseases (haemophilus influenzae type b, influenza, pertussis, pneumococcal disease and varicella zoster (chickenpox and shingles)), two of the six blood-borne diseases (hepatitis b (unspecified) and c (unspecified)), two of 11 gastrointestinal diseases (campylobacteriosis and cryptosporidiosis) and one zoonosis (leptospirosis). positive significant correlations were not observed at a lag of one month for any of the quarantinable diseases (n = 6), sexually transmissible infections (n = 6) or other bacterial infections (n = 4). it should be noted that positive significant correlations were observed at lags of over one month (but not at lag 1) for two of the top ranked 18 diseases (gonococcal infection and meningococcal disease) and 16 diseases overall (see additional file 1). additionally, the terms "haemolytic uraemic syndrome" and "leprosy" exhibited significant negative correlations with the respective disease notifications at a lag of one month. the development and application of internet-based infectious disease surveillance systems has the potential to enhance infectious disease control and prevention. whilst this is widely recognised [4, 6, 7, 12, 15, 16, 18, 20] the investigation and application of internet-based surveillance has not been systematically applied across infectious diseases; the lack of systemic knowledge regarding the potential breadth of internet-based surveillance appears to have restricted the development of systems to a small number of diseases. to our knowledge, assessments of the use of internet-based surveillance have only been performed for five of the 17 diseases that were demonstrated to have a significant association with internet search terms (influenza [4] , dengue [9, 27] , chickenpox [11, 12] , hepatitis b [14] and cryptosporidiosis [13] the authors of the final study were, however, not able to detect signals from internet search queries). our study suggests that internetbased surveillance systems have potential application to a wider range of diseases than is currently recognised. however, correlations alone should not be viewed as definitive evidence that such systems are viable; some discretion must be applied, particularly as the analyses performed were univariate. correlations between internet metrics and both gonococcal infection and chlamydia (figure 1 , boxes 2 and 7) were high; this appears to be due to a general upward trend in both and internet metrics appears to have little value in detecting perturbations in cases beyond this. this is supported by the cross correlation results (which are seasonally differenced); despite being ranked 2nd and 7th by spearman rho (figure 2 ), no positive correlations were observed for these disease/search term cross correlations, even at lag 0 ( figure 3) . further research needs to be performed; however, this study suggests surveillance systems build on internet search data to have significant promise for a number of diseases beyond those previously described, most notably pneumococcal disease, ross river virus infection, pertussis, barmah forest virus and invasive meningococcal disease. the application of internet-based data to monitoring systems of interest has been termed "nowcasting"; this approach does not predict the occurrence of future events, but rather seeks to produce more timely information on the systems of interest [28] . for infectious disease surveillance, this is typically achieved through the ability of internet-based surveillance systems to collect data at an earlier time point than is possible for traditional systems or by circumventing bureaucratic structures inherent to traditional systems that impede information flow [4] . search terms that exhibit a high level of correlation with disease notifications are of value as they may be used to provide faster intelligence on emerging disease events. results of cross correlations (figure 3) , however, indicated that forecasting of infectious disease events may also be possible using internet-based data. of the 17 diseases that exhibited significant spearman's correlations, 12 also had significant positive cross correlations at a lag of one month. overall, cross correlations indicated that forecasting of notification rates using internet-based metrics would be most realistic for the vaccine-preventable and vector-borne diseases. despite search terms offering strong or very strong correlations for two of the sexually transmissible diseases, neither exhibited significant correlations at a lag of one month. whilst internet metrics may provide valuable information regarding disease status, it is important to view these within context. the term "dengue mosquito" (figure 3 , panel 6) leads notifications by up to one month. the data imply dependence of dengue notifications on searches for the term "dengue mosquito". the mechanism of this dependence is more likely that environmental conditions that increase the abundance of mosquitos in dengue risk areas correlate with both an increase in dengue notifications and increased search interest for "dengue mosquito", allowing the search term to be used as an indicator for notifications. in this context the internet metrics also provide information that is of potential significance with respect to control of dengue fever; there is increased interest regarding mosquitos in the community and this may be driven by an increase in mosquito numbers. conversely the incidence of disease in the community may also affect search habits. the search term "chikungunya" lags notifications for chikungunya virus infection (figure 3 , panel 18). searches for "chikungunya" are probably driven by media exposure. media bias has previously been reported to adversely affect internet-based surveillance systems [27, [29] [30] [31] [32] [33] and an increase in cases of a disease in the community will likely result in the publication of stories about the disease in the media; in turn, media exposure will drive internet searches on the topic. these processes, however, are not necessarily mutually exclusive. searches for a disease may lead notifications, however, increased notifications and reporting of an emerging disease event in the media may also drive internet searches. the complexity of this relationship may make interpretation of google trends' data more difficult. for pertussis (figure 3 , (see figure on previous page.) figure 3 cross correlation results for the 18 diseases with the highest spearman's rho values . cross correlations for two search terms are displayed for each disease. coloured bars correspond to the search term with the highest spearman's rho value for each disease (red bars indicate values that exceed the 95% confidence interval, whereas blue bars do not). unfilled bars indicate cross correlation results for alternative search terms with highest cross correlation values at a lag value of 1. confidence intervals (95%) are indicated by the grey lines. panel 8), the term "whooping" exhibits a significant positive correlation with disease notifications from lag −7 through to lag 3. it appears that both mechanisms occur for the same term, demonstrating a potential difficulty in interpreting these data. it is imperative that any terms used in the development of forecasting models are heavily screened to address the complexities of the driving forces behind health-information seeking and routinely re-evaluated to account for any shifts in search behaviour which may occur [4] . there were a number of obvious limitations to this study. the temporal resolution of the data used was monthly. internet-based surveillance systems built upon monthly data are unlikely to provide better intelligence than existing traditional surveillance systems; these commonly rely upon weekly or daily reporting. this was a function of the availability of the notification data. secondly, the analyses were performed for a specific setting: australia. the nuances of language will create differences in the applicability, not just for different countries, but also within a country and between different settings (such as during an influenza pandemic) [4] . australia was selected as the study area because internet penetration in australia is very high (>80%) [34] and use is largely restricted to a single search engine; google maintains a market share of over 90% in australia [35] . these features reduce biases associated with unequal patterns of use and/or access. additionally, owing to its extensive size, australia exhibits a range of climates and varying environmental conditions, making it susceptible to a wide range of infectious diseases, including endemic and nonendemic vector-borne diseases. additionally, australia has a strong public health network and comprehensive infectious disease surveillance systems which compile high quality data on a range of diseases. combined, these features of internet usage and availability, infectious disease surveillance systems and diseases susceptibility patterns make australia an ideal system in which to study the potential application of internet-based surveillance systems. it is hoped that this work will stimulate further research into internet-based infectious disease surveillance systems beyond australia. even within our own study, however, we observed variation in correlations between internet search metrics and disease notifications for the various states ( figure 2 ). it is imperative to develop models specific to the region of interest and to assess the performance of any internet-based system against traditional surveillance data specific to the region being monitored. thirdly, this study analysed the performance of only single search terms in estimating infectious disease notifications. whilst google has not revealed the terms utilised, or the weightings applied, google flu trends is reported to incorporate around 160 search terms [36] . despite using only a single search term for each analysis, notifications for 13 diseases were identified as having a strong or very strong correlation with the selected search terms. compounding this is the fact that bonferroni adjustments were applied in assessing significance. bonferroni adjustments have previously been criticised for being overly conservative and for increasing the occurrence of type ii errors (false negatives) [25] . as such, whilst this study provides a base for future research, it would be remiss to limit future investigations to just these diseases. this study identified numerous infectious diseases of public health significance that had not previously been investigated to have potential for monitoring using internetbased surveillance systems however, this study did not seek to produce robust, accurate, internet-based surveillance systems or early warning systems that are able to produce actionable and timely data for public health units. the aim of this study was to identify the diseases for which this is possible and to focus future research efforts into these. to achieve this aim, this study used univariate analyses to determine the usefulness of internet search metrics for monitoring a wide range of infectious diseases. whilst this simplistic approach was useful for screening diseases, it will not suffice in monitoring or forecasting incidence. future studies should focus on developing composite indexes incorporate multiple search terms, or data sources (such as weather data). models built in such a manner are more resilient to media-driven behaviour, fear-based searching and evolutions in language [4] . internet-based surveillance systems have the potential to be applied to more than just enumerating disease cases within the community or predicting the onset, peak and magnitude of outbreaks. internet-based systems also have value as tools for planning emergency department staffing and surge capacity [31, 37] or for healthcare utilisation [38] . future research needs to also investigate to application of internet-based data; the greatest challenge in this field may not actually be creating models for forecasting or monitoring disease within the community, but rather applying and articulating the significance in a manner that is beneficial. internet-based surveillance systems have broader applicability for the monitoring of infectious diseases than is currently recognised. furthermore, internet-based surveillance systems have a potential role in forecasting of emerging infectious disease events. additional file 1: complete tables of results for google correlate searches, google trends data, spearman correlations and cross correlations. trends and directions of global public health surveillance modeling the effects of epidemics on routinely collected data global capacity for emerging infectious disease detection internet-based surveillance systems for monitoring emerging infectious diseases big data. the parable of google flu: traps in big data analysis google trends: a web-based tool for real-time surveillance of disease outbreaks monitoring influenza activity in europe with google flu trends: comparison with the findings of sentinel physician networks -results for 2009-10. euro surveillance: bulletin europeen sur les maladies transmissibles = european communicable disease bulletin detecting influenza epidemics using search engine query data using web search query data to monitor dengue epidemics: a new model for neglected tropical disease surveillance notifiable infectious disease surveillance with data collected by search engine more diseases tracked by using google trends diseases tracked by using google trends, spain. emerg infect dis syndromic surveillance for local outbreak detection and awareness: evaluating outbreak signals of acute gastroenteritis in telephone triage, web-based queries and over-the-counter pharmacy sales monitoring epidemic alert levels by analyzing internet search volume early detection of disease outbreaks using the internet the utility of "google trends" for epidemiological research: lyme disease as an example internet queries and methicillinresistant staphylococcus aureus surveillance norovirus disease surveillance using google internet query share data use of internet search data to monitor impact of rotavirus vaccination in the united states syndromic surveillance models using web data: the case of scarlet fever in the uk. inform health soc care digital disease detectionharnessing the web for public health surveillance tuberculosis surveillance by analyzing google trends national notifiable diseases surveillance system australian national notifiable diseases and case definitions what's wrong with bonferroni adjustments time series analysis: forecasting and control prediction of dengue incidence using search query surveillance predicting the present with google trends euro surveillance: bulletin europeen sur les maladies transmissibles = european communicable disease bulletin monitoring influenza activity in the united states: a comparison of traditional surveillance systems with google flu trends google flu trends: correlation with emergency department influenza rates and crowding metrics google watches over flu should we fear "flu fear" itself? effects of h1n1 influenza fear on ed use world telecommunication/ict indicators database statcounter global stats -top 5 seach engines in australia from assessing google flu trends performance in the united states during the 2009 influenza virus a (h1n1) pandemic using google flu trends data in forecasting influenza-like-illness related emergency department visits in omaha, nebraska. the american journal of emergency medicine using search engine query data to track pharmaceutical utilization: a study of statins the salary for gjm was provided through the australian national health the authors declare that they have no competing interests.authors' contributions gjm and wh developed the original idea for this study. development of the script for data collection was performed by smra. data analysis was performed by gjm with the assistance of wh, jsb, st and acac. the manuscript was primarily written by gjm with editorial advice from wh, smra, jsb, st and acac. all authors read and approved the final manuscript. key: cord-297835-ukrz8tlv authors: leith, douglas j.; farrell, stephen title: measurement-based evaluation of google/apple exposure notification api for proximity detection in a light-rail tram date: 2020-09-30 journal: plos one doi: 10.1371/journal.pone.0239943 sha: doc_id: 297835 cord_uid: ukrz8tlv we report on the results of a covid-19 contact tracing app measurement study carried out on a standard design of european commuter tram. our measurements indicate that in the tram there is little correlation between bluetooth received signal strength and distance between handsets. we applied the detection rules used by the italian, swiss and german apps to our measurement data and also characterised the impact on performance of changes in the parameters used in these detection rules. we find that the swiss and german detection rules trigger no exposure notifications on our data, while the italian detection rule generates a true positive rate of 50% and a false positive rate of 50%. our analysis indicates that the performance of such detection rules is similar to that of triggering notifications by randomly selecting from the participants in our experiments, regardless of proximity. there is currently a great deal of interest in the use of mobile apps to facilitate covid-19 contact tracing, see e.g. [1] [2] [3] . the basic idea of a contact tracing app is that if two people carrying mobile handsets installed with the app spend significant time in close proximity to one another (e.g. spending 15 minutes within 2 metres) then the apps on their handsets will both record this contact event. contact tracing apps based on the google/apple exposure notification (gaen) api [4] are currently being rolled out across europe, with apps already deployed in italy, switzerland and germany. these apps use bluetooth received signal strength to estimate proximity and will likely be used as an adjunct to existing manual contact tracing and test systems. existing manual systems can usually readily identify the people with whom an infected person share accommodation and with work colleagues with whom the infected person is in regular contact. more difficult is to identify people travelling on public transport with whom an infected person has been in contact, since the identities of these people are usually not known to the infected person and are generally not otherwise recorded. public transport is therefore potentially an important use case where effective contact tracing apps may be of significant assistance in infection control. a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 we report on the results of a covid-19 contact tracing app measurement study carried out on a commuter tram. the tram is of a standard design widely used in europe. measurements were collected between 108 pairs of handset locations and are publicly available [5] . in summary, our measurements indicate that in the tram there is little correlation between received signal strength and distance between handsets. similar ranges of signal strength are observed both between handsets which are less than 2m apart and handsets which are greater than 2m apart (including when handsets are up to 5m apart). this is likely due to reflections from the metal walls, floor and ceiling within the tram, metal being known to be a strong reflector of radio signals [6, 7] , and is coherent with the behaviour observed on a commuter bus [8] . we applied the detection rules used by the italian, swiss and german contact tracing apps to our measurement data and also characterised the impact on performance of changes in the parameters used in these detection rules. we find that the swiss and german detection rules trigger no exposure notifications, despite around half of the pairs of handsets in our data being less than 2m apart. the italian detection rule has a true positive rate (i.e. correct detections of handsets less than 2m apart) of around 50%. however, it also has a false positive rate of around 50% i.e. it incorrectly triggers exposure notifications for around 50% of the handsets which are greater than 2m apart. this performance is similar to that of triggering notifications by randomly selecting from the participants in our experiments, regardless of proximity. we observe that changing the people holding a pair of handsets, with the location of the handsets otherwise remaining unchanged, can cause variations of ±10db in the attenuation level reported by the gaen api. this is pertinent because this level of "noise" is large enough to potrentially have a substantial impact on proximity detection. the experimental protocol was reviewed and approved by the ethics committee of the school of computer science and statistics, trinity college dublin. the ethics application reference number is 20200503. oral consent was obtained from participants. our experimental measurements were collected on a standard light-rail tram carriage used to carry commuters in dublin, ireland, see fig 1(a) . we recruited seven participants and gave each of them google pixel 2 handsets. we asked them to sit in the relative positions shown in fig 1(b) . this positioning aims to mimic passengers respecting the relaxed social distancing rules likely during easing of lockdown and with the distances between participants including a range of values < 2m and a range of values > 2m, see fig 2. each experiment is 15 minutes duration giving around 3 scans by the gaen api when scans are made every 4 mins (per measurements reported in [9] ). a wifi hotspot was set up on the tram and the participants were asked to hold the handset in their hand and use it for normal commuter activities such as browsing the internet. after the first experiment was carried out participants were then asked to switch seats (they chose seats themselves) and a second 15 minute experiment run. after the second experiment participants were again asked to change seats for the third 15 minute experiment and, in addition, two participants were asked to place their handsets in their left trouser pocket (in an orientation of their choice). each handset had the gaen api and a modified version of the google exemplar exposure notification app [10] installed, and was registered to a gmail user included on the google gaen whitelist so as to allow use of the gaen api by the exposure notification app. each handset also had a gaenadvertiser app developed by the authors installed. this app implements the transmitter side of the gaen api and allowed us to control the tek used and also to start/stop the broadcasting of bluetooth le beacons. at the start of each 15 minute experiment participants were asked to configure the gae-nadvertiser app with a new tek and then to instruct the app to start broadcasting gaen beacons. at the end of the experiment the gaenadvertiser stopped broadcasting beacons. in this way a unique tek is associated with each handset in each experiment, and these can be used to query gaen api to obtain separate exposure information reports for each handset in each experiment. following all three experiments the handsets were collected, the teks used by each handset extracted and the gaen api on each then queried for exposure information relating to the teks of the other handsets. in total, therefore, from these experiments we collected gaen api reports on bluetooth le beacon transmissions between 108 pairs of handset locations. this measurement data is publicly available [5] . to provide baseline data on the radio propagation environment we also used the standard android bluetooth le scanner api to collect measurements of rssi as the distance was varied between two google pixel 2 handsets placed at a height of approximately 0.5m (about the same height as the tram seating) in the centre aisle of the tram carriage. we used google pixel 2 handsets running gaen api version 202512001 as reported in the settings-covid 19 notifications handset display, which includes a major update by google issued on 13th june 2020. we used a version of the google exemplar exposure notification app modified to allow us to query the gaen api over usb using a python script (the source code for the modified app is available on github [10] ). in addition we also wrote our own gaenadvertiser app that implements the bluetooth le transmitter side of the gaen api [4] . gaenadvertiser allows us to control the tek, and in particular reset it to a new value at the start of each experiment. in effect, resetting the tek makes the handset appear as a new device from the point of view of the gaen api, and so this allows us to easily collect clean data (the gaen api otherwise only resets the tek on a handset once per day). we carried out extensive tests running gaenadvertiser and the gaen api on the same device to confirm that under a wide range of conditions the responses of the gaen api on a second receiver handset were the same for beacons from gaenadvertiser and the gaen api, see [9] for further details. subsequent to our measurement study google has now published the code for the transmitter side implementation and details of the receiver side attenuation calculation [11, 12] . these also confirm that the gaenadvertiser implementation is essentially identical to the google transmitter-side implementation. gaenadvertiser is open source and can be obtained by contacting the authors (we have not made it publicly available, however, since it can be used to facilitate a known replay attack against the gaen api [13] ). the basic idea of a contact tracing app is that if two people carrying mobile handsets installed with the app spend significant time in close proximity to one another (e.g. spending 15 minutes within 2 metres) then the apps on their handsets will both record this contact event. if, subsequently, one of these people is diagnosed with covid-19 then the contact events logged on that person's handset in the recent past, e.g. over the last two weeks, are used to identify people who have been in close contact with the infected person. these people might then be made aware of the contact and advised to self-isolate or take other appropriate precautions. for this approach to be effective it is, of course, necessary that the app can accurately detect contact events. the gaen api uses bluetooth le wireless technology as the means for detecting contact events. bluetooth le devices can be configured to transmit beacons at regular intervals. to distinguish between beacons sent by different handsets each handset running gaen generates a random temporary exposure key (tek) once a day. this tek is then used to generate a sequence of rolling proximity identifiers (rpis), approximately one for each 10 minute interval during the day (so around 144 rpis are generated). the gaen system running on a handset transmits beacons roughly every 250ms. each beacon contains the current rpi value. approximately every 10 minutes the beacons are updated to transmit the next rpi value. by constantly changing the content of beacons in this way the privacy of the system is improved. in addition to the rpi each beacon also carries encrypted metadata containing the wireless transmit power level used. although beacons are emitted roughly every 250ms, on the receiving side, devices only scan for beacons roughly every 4 minutes [9] . the basic idea is that the signal strength with which a beacon is received provides a rough measure of the distance between transmitter and receiver. namely, when the received signal strength is sufficiently high then this may indicate a contact event and, conversely, when the received signal strength is sufficiently low then this may indicate that the handsets are not in close proximity. this is based on the fact that in general the radio signal gets weaker as it travels further since the transmit power is spread over a greater area. however, many complex effects can be superimposed upon this basic behaviour. in particular, obstacles lying on the path between the transmitter and receiver (furniture, walls etc) can absorb and/or reflect the radio signal and cause it to be received with higher or lower signal strength. a person's body also absorbs radio signals in the 2.4 ghz band used by bluetooth le and so the received signal strength can be substantially reduced if their body lies on the path between the transmitter and receiver. the relative orientation of two handsets can strongly affect the received signal strength owing to way antennae are packaged within the handset body. in indoor environments walls, floors and ceilings can reflect radio signals even when they are not on the direct path between transmitter and receiver, and so increase or decrease the received signal strength. see, for example, [14] for measurements illustrating such effects in real environments. the gaen system presents an interface to health authority apps this interface allows these apps to submit a request that includes an exposure configuration data structure to the gaen system [4] . the exposure configuration data structure allows specification of the tek to be queried, the start time and duration of the interval of interest (specified in 10 minute intervals since 1st jan 1970) and a low and high attenuation threshold (specified in db). the gaen system responds with one or more exposure information data structures that report an exposure duration (field durationminutes) and an array with three atttenuation duration values, giving the duration (in minutes) that the attenuation level is below the low threshold, the duration the attenuation level is between the low and high thresholds and the duration above the high threshold. it is also possible to query for an exposure summary response, but we did not make use of this since the relevant information that this contains can be derived from the exposure information reports. for each tek and time interval we made repeated queries to the gaen api holding the low threshold constant at 48db and varying the high threshold from 49db to 100db (in 1db steps up to 80db, then in 5db steps since noise tends to be higher at higher attenuation levels). by differencing this sequence of reports we can infer the attenuation duration at each individual attenuation level from 48db through to 100db. at the time of our meaurement study the gaen documentation did not precisely state how the attenuation level is calculated, nor did it give details as to how the attenuation duration is calculated. the analysis in [9] , indicated the attenuation level is calculated as p tx − p rx , where p tx is the transmit power level sent in the beacon metadata and p rx is given by a filtered rssi for google pixel 2 handsets (and others) the rssi is recorded only from beacons transmitted on one of the three radio channels used by bluetooth le for transmitting beacons, see [9] . measurements plus a calibration offset. google has subsequently published documentation [12] that confirms this. for the google pixel 2 handsets and gaen api version 202512001 used in our experiments p tx is -31db and the calibration offset is -6db. google supplied us with the calibration and offset values used for all handset models in gaen version 202512001 and we have posted these in our online study archive [5] . note that we observed that the noise floor (the rssi below which beacons can no longer be reliably decoded) is around -100db in a pixel 2, giving a maximum measureable attenuation of around 75db i.e. above this attenuation level beacons are generally not decoded successfully and so no rssi values are reported by bluetooth scans. fig 3(a) plots the attenuation measured between two handsets placed at seat height in the aisle of the tram as the distance between them is varied. these measurements were taken using the standard android bluetooth le scanner api (rather than the gaen api). this scanner api reports an rssi value for each received beacon. following [9] updated to reflect gaen calibration changes pushed by google on 13th june 2020, for the google pixel 2 handsets used in our experiments we map from rssi to attenuation level using the formula -31-(rssi-6) db. it can be seen that the attenuation initially increases as the distance is increased from 0.5m to 1.5m, as might be expected. but thereafter the attenuation level stays roughly constant with increasing distance out to 2.5m. there is then a sharp rise in the attenuation at 3m. this corresponds to the end of a group of seats and the start of a flexible joint between two carriages. as the distance is increased further it can be seen that the attenuation starts to fall. the attenuation is around 52db at 1.5m and around 60db at 4m. the baseline measurements in fig 3 indicate that the radio attenuation within the tram does not simply increase with the distance between handsets. this is similar to the behaviour observed in previous gaen measurements taken on a bus [8] , and is of course pertinent to the use of attenuation level as a proxy for distance. although further measurements are needed to confirm this, it seems likely that this effect is due to reflections from the tram walls, ceiling and floor, all of which are made of metal and highly reflective at the bluetooth radio frequency. the full attenuation duration data reported by gaen api is given in the appendix and is publicly available online [5] . in this section we analyse two aspects of this data: (i) the relationship, if any, between attenuation level and distance between handsets and (ii) the magnitude of the variations in the attenuation level induced by differences in the way participants hold their handsets. fig 4 plots the mean attenuation level vs the distance between participants in the three tests. the mean is calculated by weighting each attenuation level by the duration at that level reported by the gaen api and then summing over all attenuation levels. it can be seen that there is no clear trend in the mean attenuation level as the distance changes, with similar ranges of attenuation levels observed at all distances, except perhaps for distances below 1m where the attenuation level is more tightly clustered. the gaen api records the duration at each attenuation, and so effectively the full distribution of attenuation levels rather than just the mean. fig 5 plots the sum-duration that the measured attenuation level is below 55db, 63db, 68db and 73db. for each pair of handsets these values are the rescaled empirical cdf of the attenuation level evaluated at the specified values. recall that a typical definition of a proximity event is spending 15 minutes or more at a distance of 2m or less apart. we have therefore indicated the 2m distance with a vertical line in fig 5, and attenuation durations greater than 15 minutes by the shaded areas. for reliable detection of proximity events what one might like is that for an appropriate choice of threshold value the attenuation levels lie within the shaded area when the distance is less than 2m and outside the shaded area when the distance is greater than 2m. unfortunately we do not see such behaviour in between each of the three experiments the participants switch seats. the seat positions themselves remain the same, only the person sitting in the seat changes, allowing us to see the impact of differences in the way that each participant uses their handset. for beacons transmitted from each seat position fig 6 shows the mean attenuation level observed at the other seat positions (see the appendix for the full attenuation duration data). the attenuation level observed in test 1 is plotted vs the attenuation level observed in test 2. it can be seen that the points are clustered around the 45˚line, but variations of ±10db between the two tests are common. since the seating locations and environment within the tram are the same between experiments, participants have similar build and height and use the same model of handset, these variations can be attributed to differences in the way each particpant holds their handset and/or changes between tests in the way the same particpant holds their handset. such substantial variations in attenuation level are obviously pertinent to the use of attenuation level for proximity detection. the gaen api is intended for use by health authority covid-19 contact tracing apps [4] . when a person is found to be infected with covid-19 the teks from their handset are uploaded to a central server. the health authority app on another person's handset can then download these teks, and use them to compare against the set of beacons received by the handset. if there is a match, the attenuation duration values reported by the gaen api can then be used to estimate the risk of infection and trigger an exposure notification is this risk is sufficiently high. a typical requirement is for a person to have spent at least 15 minutes within 2m of the infected person in order to trigger an exposure notification. the mapping from gaen attenuation durations to exposure notification is therefore largely based on use of attenuation level as a proxy for proximity between handsets. switzerland deployed a covid-19 contact tracing app based on the gaen api on 26 may 2020 [15] . the documentation for this app states that it queries the gaen api with low and high attenuation thresholds of t1 = 50db and t2 = 55db and then bases exposure notifications on the quantity es = b1 + 0.5b2, where b1 is the attenuation duration below 50db reported by the gaen api and b2 is the attenuation duration between 50db and t2 [16] . an exposure notification is triggered when es is greater than 15 mins, see table 1 . germany deployed a covid-19 contact tracing app based on the gaen api on 15 june 2020 [17] . the app is open source. by inspecting the documentation and code, and querying the server api to obtain the app configuration settings this means that the app configuration table 1 . summary of detection rules studied. b1 is the the attenuation duration below threshold t1, b2 is the attenuation duration between t1 and t2, b3 is the attenuation duration below t2. can be dynamically updated. we downloaded the detection settings from the server on 21 june 2020 and they are included in the study data repository [5] , we determined that the german app follows an approach similar to the swiss app for triggering an exposure notification, but uses values t1 = 55db and t2 = 63db and an exposure duration on 10 minutes. we applied the swiss and german exposure notification rules to the tram dataset. fig 7( a) plots the true and false positive rates for t1 = 50db and as t2 is varied from 55db upwards and the es threshold varied from 10 minutes to 15 mins. the mean rates are shown with one standard deviation indicated by the error bars. the mean and standard deviation are obtained by a standard bootstrapping approach the dataset was resampled with replacement n = 1000 times, the exposure notification percentage calculated for each sample and then the mean and standard deviation of these n estimates calculated. we selected n by calculating the mean and standard deviation vs n and selecting a value large enough that these were convergent. fig 7(b) plots the true and false positive rates when t1 = 55db. it can be seen from fig 7(a) that selecting t1 = 50db and t2 = 55db (the values used in the swiss app) yields no positive detections, despite approximately 50% of the handset pairs in the tram dataset being within a 2m distance of one another. increasing t2 to 62db and above yields a small increase in detection rate, with true and false detection rates roughly equal (we comment further on the implications of this below). it can be seen from fig 7(b) that selecting t1 = 55db and t2 = 63db (the values used in the german app) there are are no detections when the threshold for es is 15 minutes but when a threshold of 10 minutes is used, as in the app, then the true and false positive detection rates both rise to 9%. increasing t2 does not increase these detection rates. italy deployed a covid-19 contact tracing app based on the gaen api on 1 june 2020 [18, 19] . the app is open source. by inspecting the documentation and code, and querying the server api to obtain the app configuration settings we downloaded the detection settings from the italian app server on 21 june 2020 and they are included in the study data repository [5] , we determined that the app follows a different approach to the swiss and german apps, triggering an exposure notification whenever the attenuation duration is above threshold t2 = 73db i.e. without the weighting of 0.5 used in the swiss and german exposure notification rules. we applied this exposure notification rule to the tram dataset. fig 8(a) plots the true and false positive rates as threshold t2 is varied from 55db upwards and the threshold for es is varied from 10 minutes to 15 mins. for t2 = 73db the true and false positive detection rates are both around 50% when the threshold for es is 15 minutes, rising to 80% when the threshold for es is reduced to 10 minutes. as noted in section 2.5, with the calibration values used in the gaen api the maximum observable attenuation level with google pixel 2 handsets is around 75db (above this level beacons are generally no longer successfully received). selecting t2 = 73db therefore means that almost the full range of possible attenuation levels will trigger an exposure notfication. high detection rates are therefore unsurprising, but the detection has little discrimination and essentially would trigger exposure notifications for all participants in our tests regardless of proximity. fig 8(a) also shows the true and false positive detection rates for other choices of threshold t2. while the detection rates are generally substantially higher than with the swiss and german detection rules, it can be seen that the false positive rate increases almost exactly in line with the true positive rate. this can be seen more clearly when this data is replotted in roc format, see fig 8(b) . it can be seen that the true vs false positive curve lies close to the 45˚line (to avoid clutter we do not plot the error bars from fig 8(a) on fig 8(b) but the small deviation from the 45˚line is not statistically significant.). that is, the detection performance is poor, and comparable to simply selecting from participants at random when making exposure notifications. a limitation of this study is that it is confined to handsets using the android operating system. the gaen api is also implemented on apple ios devices, but apple have severely limited the ability of testers to make measurements (each handset is limited to querying the gaen api a maximum of 15 times a day, and apple has no whitelisting process to relax this constraint. our measurement approach uses 34 queries to extract fine-grained attenuation data per pair of handset locations. we equipped participants with the same model of handset in order to remove this as a source of variability in the data and instead focus on variability caused by the radio environment and the way that people hold their handsets. google and apple are currently undertaking a measurement campaign to select calibration values within the gaen api with the aim of compensating for differences between handset models. we therefore expect that our measurements should also be applicable to a range of handsets, although this remains to be confirmed. with regard to calibration, we note that bluetooth received signal strength is affected by several factors including (i) differences between different models/makes of handset, (ii) fluctuations in the relative orientation of handsets (even small changes can have a large impact), (iii) absorption by human bodies (especially when phone is in a pocket), bags etc, (iv) radio wave reflection from walls, floors, furniture. see [14] for measurements highlighing the potential for significant impact of these factors. calibration may mitigate (i), although this remains unclear at present and variations between handsets might be expected to degrade performance compared to our measurements, but not (ii)-(iv). in both the tram measurements reported here and previous measurements in a commuter bus [8] only a weak correlation between received signal strength and distance between handsets is observed. a direct comparison of detection accuracy in these two datasets is unfortunately not possible since in the bus measurements all pairs of handsets were within 2m of one another and so only the rate of false negatives can be evaluated. we report on the results of a covid-19 contact tracing measurement study carried out on a commuter tram in dublin, ireland. our measurements indicate that in the tram there is little correlation between received signal strength and distance between handsets. we applied the detection rules used by the italian, swiss and german apps to our measurement data and also characterised the impact on performance of changes in the parameters used in these detection rules. we find that the swiss and german detection rules trigger no exposure notifications on our data, while the italian detection rule generates a true positive rate of 50% and a false positive rate of 50%. our analysis indicates that the performance of such detection rules is similar to that of triggering notifications by randomly selecting from the participants in our experiments, regardless of proximity. bin, with the rest of the time roughly evenly split between 62-64db, 66-68db, 70-72db. the weighted average attenuation is 70db. plot the exposure information between each pair of handsets reported by the gaen api for each of the three experiments. to assist with interpreting the plots the reports in each plot are ordered by increasing distance between the pairs of participants (see fig 1(a) ). no data is shown when no beacons were received between a pair of handsets, e.g. between particpants 2 and 3 in fig 9(b) and 9(c). it can be seen that occasionally there is an increasing trend in attenuation, for example see figs 9(c) and 11(c), but this is infrequent. occasionally there is a decreasing trend in attenuation, for example see figs 9(e) and 11(d). overall, however no consistent trend is evident in the change in attenuation level with increasing distance. in fig 11 participants 1 and 3 place their handsets in their left trouser pocket rather than their hand. intuitively, one might expect this change to increase the attenuation level since the particpants body is now more likely to affect transmission and reception of radio signals. however, comparing fig 11(a) and 11(c) with figs 9 and 10 it can be seen that this change does not cause any consistent change in the observed attenuation level. for example, comparing figs 11 (a) and 10(a) the attenuation level between participants 1 and 5 decreases from test 2 to test 3, while the attenuation level between participants 1 and 3 increases. quantifying sars-cov-2 transmission suggests epidemic control with digital contact tracing eu urges vigilance to avoid coronavirus second wave apple and google partner on covid-19 contact tracing technology exposure notifications: android api documentation dublin luas tram gaen attenuation durations dataset experimental study of propagation characteristics for wireless communications in high-speed train cars experimental characterisation and modelling of intra-car communications inside highspeed trains. iet microwaves, antennas and propagation measurement-based evaluation of google/apple exposure notification api for proximity detection in a commuter bus verifying the google/apple covid exposure notification api modified exposure notification app exposure notifications: source code snippets exposure notifications: exposure notifications ble attenuations a coronavirus contact tracing app replay attack with estimated amplification factors coronavirus contact tracing: evaluating the potential of using bluetooth received signal strength for proximity detection coronavirus: first google/apple-based contact-tracing app launched dp-3t exposure score calculation summary accessed 14 open source project; accessed 23 immuni app web site; accessed 23 immuni apple store version history the authors would like to extend their thanks to the irish health & safety executive (hse) for arranging with google for us to have whitelisted access to the gaen api, and to transdev dublin light rail for kindly providing access to one of their trams. we emphasise that any views expressed in this report are the authors own, and may not be shared by the hse, transdev or trinity college dublin. we present the full attenuation duration data using a coloured heatmap. we split the range of attenuation values into 2db bins, i.e. 70-72db, 72-74db and so on, up to 80db when 5db bins are thereafter used since the data is noisier at these low signal levels. within each bin the colour indicates the percentage of the total duration reported by the gaen api that was spent in that bin, e.g bright green indicates that more than 90% of the time was spent in that bin. the mapping from colours to percentages is shown on the righthand side of the plot. bins with no entries (i.e. with duration zero) are left blank. where appropriate we also include a solid line in plots that indicates the average attenuation level at each transmit power level (the average is calculated by weighting each attenuation level by the duration at that level and then summing over all attenuation levels).for example, in fig 9(a) the left-hand heatmap shows the attenuation durations measured between participants 1 and 4. the attenuation spends around 60% of its time in the 74-76db conceptualization: douglas j. leith, stephen farrell. key: cord-351448-jowb5kfc authors: ganesh, ragul; singh, swarndeep; mishra, rajan; sagar, rajesh title: the quality of online media reporting of celebrity suicide in india and its association with subsequent online suicide-related search behaviour among general population: an infodemiology study date: 2020-08-29 journal: asian j psychiatr doi: 10.1016/j.ajp.2020.102380 sha: doc_id: 351448 cord_uid: jowb5kfc the literature reports increased suicide rates among general population in the weeks following the celebrity suicide, known as the werther effect. the world health organization (who) has developed guidelines for responsible media reporting of suicide. the present study aimed to assess the quality of online media reporting of a recent celebrity suicide in india and its impact on the online suicide related search behaviour of the population. a total of 200 online media reports about sushant singh rajput’s suicide published between 14(th) to 20(th) june, 2020 were assessed for quality of reporting following the checklist prepared using the who guidelines. further, we examined the change in online suicide-seeking and help-seeking search behaviour of the population following celebrity suicide for the month of june using selected keywords. in terms of potentially harmful media reportage, 85.5% of online reports violated at least one who media reporting guideline. in terms of potentially helpful media reportage, only 13% articles provided information about where to seek help for suicidal thoughts or ideation. there was a significant increase in online suicide-seeking (u = 0.5, p < 0.05) and help-seeking (u = 6.5, p < 0.05) behaviour after the reference event, when compared to baseline. however, the online peak search interest for suicide-seeking was greater than help-seeking. this provides support for a strong werther effect, possibly associated with poor quality of media reporting of celebrity suicide. there is an urgent need for taking steps to improve the quality of media reporting of suicide in india. suicide is a major public health problem, and is one of the leading causes of mortality globally (naghavi, 2019) . the reported deaths due to suicide in india is highest among countries worldwide (dandona et al., 2018) . studies have reported that media reports of celebrity suicide stimulate imitation acts in vulnerable population (gould et al., 2003) . also, repeated insensible media coverage may act as a source of misinformation that suicide is an acceptable solution to ongoing difficulties in life. this has been supported by the bulk of available literature, with a recent meta-analysis reporting a 13% increased risk of suicide (95% confidence interval of 8-18%; median follow-up duration of 28 days) in the period following the media report of celebrity suicide death (niederkrotenthaler et al., 2020) . media reporting of suicide is a double-edged sword, with inappropriate and sensational reporting of suicide news leading to copycat phenomenon or werther effect. whereas, sensible media reporting of suicide along with media involvement in spreading preventive information shown to minimise copycat eff ects, and has been shown to be eff ective in reducing suicide deaths (cheng et al., 2018) . thus, researchers have advocated for a more responsible descriptive reporting of suicide news, with emphasis on sharing preventive information related to suicide. this includes reporting upon how people could adopt alternative coping strategies to deal with life stresses or depressed mood along with sharing links of educative websites or suicide helplines; and has been shown to be associated with decreased suicide suicidal behaviour and ideation in vulnerable population (niederkrotenthaler et al., 2010; till et al., 2017) . therefore, media reporting of suicide-related preventive information has been associated with positive effects on subsequent suicide rates and ideation. this is described as the papageno effect, and acts as a counterforce to the werther effect responsible media reporting of suicide is considered as the best available strategy to counter the harmful effects of media reportage (niederkrotenthaler et al., 2020) . in recent years, internet is being increasingly used by the public for seeking health-related information; and information related to mental health related disorders or problems also being widely available online (amante et al., 2015) . it is understandable, that several researchers have expressed concerns about vulnerable individuals either using internet to access pro-suicide information (e.g. methods of suicide) or inadvertently being exposed to online news or information which negatively affects their thoughts or mood and promote suicidal behaviours in them (arendt and scherr, 2017; till and niederkrotenthaler, 2014) . however, internet also provides a host of suicide prevention related information and resources which could in turn decrease the risk of suicide (biddle et al., 2008) . further, news over internet and social media is able to reach to a large number of vulnerable and difficult to reach youth population; and has been shown to potentially influence the public opinion, attitudes, and behaviours over wide range of topics (kaplan and haenlein, 2010) . thus, it is important to explore the quality of online media reporting of celebrity suicide in india. this would in turn help in better understanding the role played by this new electronic medium in either predisposing or protecting people with suicidal ideas or death wishes. there has been limited literature available assessing the quality of media reporting of suicide in india. most of the studies assessed media reporting of suicide in general population, and only one study had focused on celebrity suicide specifically (harshe et al., 2016) . however, that study took death of robin williams (hollywood movie actor of us origin) as the reference event and was done about four years back. further, all the available studies have assessed newspapers in a particular region and were conducted prior to press council of india (pci) issuing media reporting guidelines on suicide and mental illness in india. the pci has adopted the guidelines of world health organization (who) report on preventing suicide (press council of india, 2020). it forbids undue repetition of stories, placing stories in prominent positions, explicit description of the suicide method, providing details about the suicide location, using sensational headlines and reporting photographs of the person. there might be some change in the quality of media reporting of suicide in recent years, more specifically after the pci guidelines. further, the who guidelines for responsible reporting are valid for all types of media, and it is important to explore the role played by the online media in current digital world. moreover, to the best of our knowledge there has been no study from indian context yet exploring the association between media reporting of death of celebrity by suicide and subsequent suicidal behaviour in the general population. the official figures for deaths due to suicide in india is released by the national crimes record bureau (ncrb) in india. however, the ncrb has stopped releasing this data since 2016 and the official suicide statistics have not been made public till now. further, it is usually available at the end of the year and does not provide data on a weekly basis. moreover, any other system of directly recording suicide statistics in india will face the challenges associated with collecting vital statistics through sub-optimal existing vital registration system, misclassification, and under-reporting of suicide deaths due to associated legal complications and social stigma around suicide death in the family . additionally, the restrictions imposed on movement of people and social distancing guidelines to be followed during the current covid-19 pandemic, makes it even more difficult to access the study population in a systematic manner for assessment of suicide risk using traditional research methods (bidarbakhtnia, 2020) . the above described limitations could be addressed by employing research methods and techniques involving the study of internet-based search behaviours and social media content. infodemiology has been defined as "the science of distribution and determinants of information in an electronic medium, specifically the internet, or in a population, with the ultimate aim to inform public health and public policy" (eysenbach, 2009) . google trends is an analytical tool available for tracking the online search interests of the population. the evidence supporting correlation between increased online search interest for particular suicide-related search queries using google search engine and the actual number of suicides in that region during that particular time-period has been increasing over the past two decades (lee, 2020) . moreover, recent studies have shown that data obtained using google trends for suicide-seeking keywords could be used for predicting actual monthly suicide numbers at the country level (kristoufek et al., 2016) . thus, in the present study we monitored the changes in internet search volumes for keywords representing suicide-seeking and help-seeking behaviours using the google trends platform as a proxy marker to assess the impact of recent celebrity suicide in india. sushant singh rajput (ssr) was a much-loved indian actor who died by suicide on june 14, 2020. this was reported by various national and international media, and was considered as the reference event in this study. thus, the present study aimed to assess the quality of online media reporting of a celebrity suicide in india, and evaluate its adherence with the who guidelines for responsible media reporting of suicide. further, we aimed to examine the change in internet search volumes for keywords representing suicide-seeking and help-seeking behaviours of the population immediately following the celebrity suicide. this would provide indirect evidence for either existence or absence of the werther and the papageno effect at the population level in india. the online media reports related to the theme of death of ssr by suicide on 14 th june were retrieved using the google news online platform (https://news.google.com). the search was conducted on 20 th june, 2020 in the tor browser, using search terms "sushant", "singh", "rajput", "died", "death" and "suicide". the search period was restricted between 14 to 20 june, 2020. this corresponded to first week immediately after the reference event. a total of 214 reports published on various international, national, and regional online news and entertainment media portals were retrieved. fourteen of them contained either only videos or were not related to theme of the present study, and were excluded. thus, a total of 200 articles were selected for further analysis. the news headlines were analysed to generate a word cloud (using a word cloud generator available at https://www.wordclouds.com) representing the commonly used terms in the online media reports covering ssr death. two authors independently reviewed and extracted information related to different news report characteristics using a pre-designed format in microsoft excel. it included information pertaining to descriptive characteristics of the news report such as the date of publishing, name of the news publisher, type media agency, and primary focus of the article being descriptive or commentary. the quality of articles was evaluated using a checklist prepared on the basis of who j o u r n a l p r e -p r o o f responsible media reporting of suicide guidelines (see supplementary table 1) , and is similar to that used in previous studies . the items were coded as "1" if the guideline was violated and "0" if the guideline was adhered to in the report. two trained researchers independently reviewed and extracted information following the above described procedure. any discrepancy or disagreement between the two researchers was resolved by consensus. the third author was consulted if needed. the data were analysed using spss version 23.0 (ibm corp, armonk, ny). the descriptive (frequency and percentage) and inferential statistics (chi-square and fishers' exact test) were conducted. a p-value of less than 0.05 was considered significant for all tests. the google trends utilizes an algorithm to give normalized relative search volume (rsv) for the keyword(s) searched for a specified geographical region and time period. the rsv represents how frequently a given search query has been searched on the google search engine, compared to the total number/volume of google searches conducted in the same geographical region over the selected time period. the rsv values range from zero (representing very low search volumes) to 100 (peak search volume for that query). google trend analysis was conducted to evaluate the online search interest for keywords representing suicide-seeking and help-seeking behaviours of the population for the month of june 2020. the initial list was made based on the review of available literature, which was finalized by the process of consensus building between two authors r.g. and s.s (qualified psychiatrists with clinical and research experience of working with people with mental illness and suicidal ideas/attempts) based on the face validity of search terms. the examples of suicide-seeking keywords included in the study were 'commit suicide', 'suicide method', and 'kill myself'. whereas, the helpseeking keywords such as 'suicide help', 'suicide treatment', and 'psychiatrist' were used. the four google trends options of region, time, category, and search type were specified as india, from 1 june to 30 june 2020, all categories, and web search in the present study. the "plus" (+) function from google trends was used to integrate the search volume (rsv) of all suicide-seeking terms and help-seeking keywords. a graph showing daily variation in rsv for suicide-seeking and helpseeking keywords was constructed. the change in mean rsv value for the suicide-seeking and helpseeking keywords after the reference event, when compared to baseline was analysed by applying the mann whitney-u test. the complete list of keywords used in this study along with other details pertaining to the google trends methodology are described in supplementary table 2. the information used in this study involved published online media reports and data related to the volume of anonymized web searches made during a given time period, both of which were freely available in the public domain. further, no patient or participant was approached directly in this study. thus, no written ethical permission was required from the ethics committee. the frequency of different words used in the headlines of the 200 media reports analysed in the present study were depicted as a word cloud, with the size of font used being representative of its frequency (figure 1) . apart from the words in the name of ssr, the most commonly used words were "suicide", "death", "actor", "police", "bollywood", "mumbai", "rhea", and "kapoor" in decreasing order of frequency. this suggested that a significant proportion of headlines used words like suicide, police or bollywood to sensationalize or glamourize the headlines, with no significant difference between news media (37.5%; 37/122) and entertainment media (42.0%; 37/88) headlines (χ 2 =0.426, p=0.56). the term "suicide" was used with similar frequency in both news media (n=18;16.1%) and entertainment media (n=15;17%) headlines. only two news media reports (n=2;1.8%) mentioned 'hanging' term in the headlines. the location of suicide was mentioned in two news media (1.8%) and two (2.3%) entertainment media headlines. the selected media reports were published from various media platforms: international news group, 9% (n=18); national news group, 28.5% (n=57); regional news group, 18.5% (n=37); and entertainment blogs, 44% (n=88). seventeen news media platforms had reported the story four or more times in the immediate one-week period following the ssr suicide, with hindustan times (6), ndtv (6), republic world (6), times of india (5), dna (5), india tv (4), the indian express (4) and times now (4) contributing to 41.5% (n=83/200) of the articles. around 17% (n=34) articles were published on 14 th june, 2020, while 49.5% of articles (n=99) were published on 18 th june,2020. about 45.5% (n=91/200) articles were focussed at direct descriptive reporting of suicide. the descriptive analysis of media reports for different potentially harmful and helpful media report characteristics are described in table 1 and 2 respectively. about 85.5% of reports violated the recommendation provided in the guideline, by including at least one potentially harmful information. there was significant association between the type of news media and the use of sensational language [χ 2 (3)=9.774 (p<0.021)]. regional and entertainment media used more sensational language compared to national and international media. there was significant association between the type of news media and provision of information about where to seek help [χ 2 (1)=15.98 (p<0.001)]. mainstream news media provided such information more than entertainment media. final social media posts were shared more by national media compared to international, regional and entertainment media [χ 2 (3)=9.91 (p<0.019)]. the median and inter-quartile range (iqr) values of rsv for suicide-seeking keywords in the twoweeks before and after the death of ssr on 14 june 2020 were 12 (iqr: 11.5-13.5) and 22 (iqr: 18-25) respectively. whereas, the median and iqr values of rsv for suicide-seeking keywords in the two-weeks before and after the death of ssr were 11 (10-11.5) and 14 (13-19) respectively. there was a significant increase in rsv for suicide-seeking (u=0.5; z= -4.62; p<0.001) and help-seeking (u=6.5; z=-4.39; p<0.001) keywords after the reference event, when compared to baseline. however, the online peak search volume and search interest for suicide-seeking was greater than help-seeking as shown in figure 2 . the present study analysed the online media reports related to the theme of a popular bollywood movie actor's suicide, and compared it against the who media reporting guidelines for suicide. the story of this recent celebrity suicide received widespread coverage across different online news platforms, including national and international news agencies. overall, majority of articles showed poor adherence with the who guidelines while reporting the celebrity suicide. the reports had minimal focus on educating the public the regarding suicide. further, the change in online search interest for different keywords related to "suicide-seeking" and "help-seeking" behaviours after this event were analysed to explore for possible werther and papageno effects. a substantial proportion of articles did not follow most of the recommendations. about 58% articles used sensational language, 59.5% articles mentioned suicide site, 17% articles suggested possible cause for suicide which was not related to poor mental health. a study assessing the quality of suicide reporting in indian print media found increase in prominence of suicide reports after the celebrity suicide (harshe et al., 2016) . it speculated that the most likely reason for sensationalism in media reporting of suicide might be to enhance the readership. further, only 13% articles provided information about where to seek help for suicidal thoughts. a previous study evaluating the newspaper coverage of celebrity suicide in united states against 'mindset' recommendations for reporting suicide, found 46% articles provided details about suicide method and only 11% provided information about help-seeking (carmichael and whitley, 2019) . previous studies from india found minimal adherence to media reporting recommendations for suicide in the print media. menon et al found that the method of suicide was reported in 99.1% articles and locations of suicide was reported in 86.5% articles (menon et al., 2020) . chandra et al showed that 80% articles reported suicide location and 61% suggested monocausality for suicide (chandra et al., 2014) . the high frequency of harmful reporting characteristics observed in the present study is consistent with the low adherence to who guidelines reported in other neighbouring asian countries as well (s.m. yasir . studies from bangladesh (s. m. yasir , indonesia (nisa et al., 2020) and sri lanka (brandt sørensen et al., 2019) have also reported non-adherence to who recommendations in print media such as reporting of suicide method, description of suicide note and inclusion of personal identification characteristics in reports. the headlines of the online reports included in the current study used the term 'suicide' in 18% articles. previous studies on print media from india reported "suicide" mentioned in headlines of 68.6% articles (chandra et al., 2014) and 30.31% articles (harshe et al., 2016) . refreshingly, in the present study 81.5% articles did not use the word 'commit' or related terms while reporting suicide, and 65.5% articles did not mention the suicide method in the reports. this is a welcome improvement in media reportage of suicide, which might be due to the positive effect of pci adopting guidelines on media reporting of suicide in september 2019 based on the who guidelines (vijayakumar, 2019) . however, photograph of the celebrity was provided by 47.5% of news media reports and 38% of entertainment media reports. publication of photograph of a person with mental illness without the individual's or his/her next of kin's consent in case of suicide violates section 24 (1) of the mental health care act, 2017 in india ("mental healthcare act," 2017). further, sensational language was used to report celebrity suicide by majority of news media and entertainment media reports. among the news media, regional media used sensational language more frequently than national and international media. final social media posts were reported by 8.9% news media and 3.4% entertainment media. among the different media platforms, national media shared final social media posts more frequently than international, regional and entertainment media. mainstream news media provided information about where to seek help more frequently than entertainment media. this is in line with a study on print media from india, that reported vernacular newspapers to be more compliant with who suicide reporting guidelines compared to english language newspapers (menon et al., 2020) . moreover, in terms of providing potentially helpful information, only one article provided research findings and population level statistics regarding suicide. only two percent articles included expert opinion from health professionals while reporting suicide. also, 28.5% articles tried to address the link between suicide and poor mental health in the present study. this highlights the need to emphasize the importance of including such information in media reports of suicide among journalists and news editors. it helps in increasing the awareness about mental health problems among the general population and encourage them to seek treatment for the same. a previous study assessing south indian newspapers found that a few articles recognised the link between suicide and psychiatric disorders or substance use disorders (menon et al., 2020) . similarly, previous studies from india evaluating the reporting of suicides in newspapers found that only few articles tried to educate public about the issue of suicide by including opinion from health professionals, research findings or information about suicide prevention programmes (chandra et al., 2014; harshe et al., 2016; menon et al., 2020) . one possible solution is to have a uniform national suicide reporting guideline for the media of the entire country. a similar approach has been shown to be beneficial in improving the overall quality of media reporting of suicide in australia (pirkis et al., 2009) . however, as prior researchers have pointed out (vijayakumar, 2019) , merely framing of guidelines may not help in improving the quality of media reporting of suicide. a continuous collaborative approach involving both mental health experts and media professionals should be adopted to sensitize them about the available research evidence backing these media reporting guidelines has been shown to successful in improving adherence to media reporting guidelines (bohanna and wang, 2012) . also, there should be regular workshops held for media professionals to provide them with adequate training and support in covering mental health and suicide-related topics. the findings from google trend analysis showed a significant increase in online search interest for terms representing both suicide-seeking and help-seeking behaviours after the ssr death. the surge in internet search volume for suicide-seeking keywords along with media reports of copycat suicides from different parts of india provides evidence of the werther effect (hindustantimes, 2020; news18, 2020, p. 18; timesofindia, 2020) . there are several possible mechanisms described in the literature to explain the observed increase in suicidal behaviour among the general population associated with media reporting of celebrity suicide (niederkrotenthaler et al., 2020) . first, people may identify with the deceased celebrities, which is usually more common in case of entertainment celebrities due to their strong public identity and following. second, repeated insensible media reporting might lead to normalization of suicide as an acceptable way out of their problems by the vulnerable population. third, media reporting about the method of celebrity suicide might increase the cognitive availability of that method and remove ambivalence about which method to choose for suicide in vulnerable individuals, leading to an increase in suicide by the same method among the vulnerable population. interestingly, there was also smaller but significant increase in the internet search volume for helpseeking keywords. the peak search volume for help-seeking and suicide-seeking keywords was observed on the day of ssr's death, with a lower peak rsv and subsequently lower daily rsvs for help-seeking terms as compared to suicide-seeking terms among the general population. this suggests a weaker papageno effect as compared to the werther effect, possibly due to poor adherence to the who suicide reporting guidelines by the online and other types of media in india while covering the celebrity suicide (newslaundry, 2020) . there only a few studies that have assessed the fidelity of suicide reporting in india, with almost of the studies having evaluated the quality of media reporting of suicide in general population and included only few print media newspapers. thus, our study provides valuable addition to bridge these gaps in the existing literature on media reporting of celebrity suicide from india. further, a wide range of online media reports were analysed in this study for the first time in indian settings to the best of our knowledge. further, the use of a novel google trends analysis to show an increased online search interest for suicide-seeking keywords immediately after the reference celebrity suicide provided support for the existence of werther effect in the indian context. however, there are certain limitations as well which should be kept in mind while interpreting the findings of this study. the study focussed only on english language media reports. we did not assess print media without online version, television, radio and social media. this might be an important area for future research, since studies from western countries suggest television coverage or social media (e.g. twitter) to be associated with increased suicide rates (jashinsky et al., 2014) . further, the relationship between people searching for suicide-seeking keywords might not be as clear as that observed for people with certain infectious disease like the influenza, with google trends analysis of data about searching for disease symptoms or other disease-related information being used to predict their incidences or outbreaks prior to the traditional methods of reporting an outbreak (cao et al., 2017; ginsberg et al., 2009; zhang et al., 2018) . this is likely due to the fact that that someone who searched about suicide might not be actually suicidal, and may or may not kill themselves during the specified study period. further, the keywords representing suicide-seeking and help-seeking behaviours were derived from review of literature from western countries mostly followed by consensus amongst the authors based on their face validity. however, the search methodology used for doing the google trends analysis in the present study is in accordance with the guidelines for conducting a robust google trends research (nuti et al., 2014) . the quality of media reporting of celebrity suicide on online media in india is poor when compared to adherence with the who guidelines or the pci guidelines. in terms of including potentially harmful information, about 85.5% of reports violated at least one recommendation provided in the guideline. further, compliance with recommendations of including potentially helpful information about creating awareness about suicide and possible ways of seeking help for suicidal thoughts was very low, with only few articles 13% articles providing information about where to seek help for suicidal thoughts or ideation. there was a significantly greater increase in the online search interest for suicide-seeking keywords after the recent celebrity suicide. this in turn provides support for a strong werther effect, possibly associated with poor quality of media reporting of celebrity suicide. the results emphasize the need for an increased collaboration, promotion, and advocacy by experts for uptake of existing media reporting guidelines on suicide by journalists and other stakeholders. there is an urgent need for research on understanding the effects of media reporting of suicide at general population's suicidal acts and thoughts. funding sources: no financial support was received for this study. the authors declare no conflict of interest. j o u r n a l p r e -p r o o f access to care and use of the internet to search for health information: results from the us national health interview survey quality of media reporting of suicidal behaviors in south-east asia do bangladeshi newspapers educate public while reporting suicide? a year round observation from content analysis of six national newspapers optimizing online suicide prevention: a search engine-based tailored approach suicide in india: a complex public health tragedy in need of a plan assessing the quality of media reporting of suicide news in india against world health organization guidelines: a content analysis study of nine major newspapers in tamil nadu surveys under lockdown; a pandemic lesson suicide and the internet media guidelines for the responsible reporting of suicide: a review of effectiveness self-harm and suicide coverage in sri lankan newspapers social media engagement and hiv testing among men who have sex with men in china: a nationwide cross-sectional survey media coverage of robin williams' suicide in the united states: a contributor to contagion? plos one 14, e0216543 do newspaper reports of suicides comply with standard suicide reporting guidelines? a study from bangalore, india the role of media in preventing student suicides: a hong kong experience gender differentials and state variations in suicide deaths in india: the global burden of disease study infodemiology and infoveillance: framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the internet detecting influenza epidemics using search engine query data media contagion and suicide among the young celebrity suicide and its effect on further media reporting and portrayal of suicide: an exploratory study sushant singh rajput's fan ends life in visakhapatnam: report tracking suicide risk factors through twitter in the us users of the world, unite! the challenges and opportunities of social media estimating suicide occurrence statistics using google trends search trends preceding increases in suicide: a cross-correlation study of monthly google search volume and suicide rate using transfer function models do tamil newspapers educate the public about suicide? content analysis from a high suicide union territory in india global, regional, and national burden of suicide mortality 1990 to 2016: systematic analysis for the global burden of disease study andaman's minor girl hangs herself after she went into depression over sushant singh rajput's suicide [www document are top indian newspapers complying with guidelines on suicide reporting? association between suicide reporting in the media and suicide: systematic review and meta-analysis media and suicide : international perspectives on research, theory, and policy role of media reports in completed and prevented suicide: werther v. papageno effects indonesian online newspaper reporting of suicidal behavior: compliance with world health organization media guidelines the use of google trends in health care research: a systematic review changes in media reporting of suicide in australia between 2000/01 and 2006/07 guidelines adopted by the pci on mental illness/reporting of suicide cases surfing for suicide methods and help: content analysis of websites retrieved with search engines in austria and the united states beneficial and harmful effects of educative suicide prevention websites: randomised controlled trial exploring papageno v. werther effects patna: girl hangs self after watching sushant singh rajput's suicide news | patna news -times of india media matters in suicide -indian guidelines on suicide reporting using internet search data to predict new hiv diagnoses in china: a modelling study j o u r n a l p r e -p r o o f key: cord-232959-jcnvnn2k authors: arnal, raquel p'erez; conesa, david; alvarez-napagao, sergio; suzumura, toyotaro; catala, mart'i; alvarez, enric; garcia-gasulla, dario title: private sources of mobility data under covid-19 date: 2020-07-14 journal: nan doi: nan sha: doc_id: 232959 cord_uid: jcnvnn2k the covid-19 pandemic is changing the world in unprecedented and unpredictable ways. human mobility is at the epicenter of that change, as the greatest facilitator for the spread of the virus. to study the change in mobility, to evaluate the efficiency of mobility restriction policies, and to facilitate a better response to possible future crisis, we need to properly understand all mobility data sources at our disposal. our work is dedicated to the study of private mobility sources, gathered and released by large technological companies. this data is of special interest because, unlike most public sources, it is focused on people, not transportation means. i.e., its unit of measurement is the closest thing to a person in a western society: a phone. furthermore, the sample of society they cover is large and representative. on the other hand, this sort of data is not directly accessible for anonymity reasons. thus, properly interpreting its patterns demands caution. aware of that, we set forth to explore the behavior and inter-relations of private sources of mobility data in the context of spain. this country represents a good experimental setting because of its large and fast pandemic peak, and for its implementation of a sustained, generalized lockdown. we find private mobility sources to be both correlated and complementary. using them, we evaluate the efficiency of implemented policies, and provide a insights into what new normal means in spain. the covid-19 pandemic is changing the world in unprecedented and unpredictable ways. human mobility is at the epicenter of that change, as the greatest facilitator for the spread of the virus. to study the change in mobility, to evaluate the efficiency of mobility restriction policies, and to facilitate a better response to possible future crisis, we need to properly understand all mobility data sources at our disposal. our work is dedicated to the study of private mobility sources, gathered and released by large technological companies. this data is of special interest because, unlike most public sources, it is focused on people, not transportation means. i.e., its unit of measurement is the closest thing to a person in a western society: a phone. furthermore, the sample of society they cover is large and representative. on the other hand, this sort of data is not directly accessible for anonymity reasons. thus, properly interpreting its patterns demands caution. aware of that, we set forth to explore the behavior and inter-relations of private sources of mobility data in the context of spain. this country represents a good experimental setting because of its large and fast pandemic peak, and for its implementation of a sustained, generalized lockdown. we find private mobility sources to be both correlated and complementary. using them, we evaluate the efficiency of implemented policies, and provide a insights into what new normal means in spain. covid-19 has produced an unprecedented change in mobility [1, 2] . governments have implemented restrictive measures to contain the pandemic, focused on reducing social contacts [3] . this is a direct effort towards controlling the covid-19 spread, at a stage where the test and trace strategy is impractical [4] . mobility restriction measures range from the closure of schools and large gatherings, to the complete lockdown of the population and economic hibernation. different countries have implemented different measures, varying their severity and duration [5] . lifting these restrictions will lead societies to a new normality, which is still unclear how much will resemble the old normality. at this point it seems likely that societies will transition to a new basal mobility structure [6] . one we need to analyze and understand the sooner the better. there are many public sources of mobility data which can be used to measure the physical distancing of population during the covid-19 crisis [7] [8] [9] . unfortunately, most of these sources provide limited insights, focusing on a movement modality (e.g., public transport occupancy, public bike system usage, road densities). an alternative to public data is the data provided by private technological entities, which have access to a high volume of information through applications installed in mobile phones [10] . even though the segment of population that has these applications installed is not a perfect reflection of society (e.g., young segments of society are over-represented while elders and kids are under-represented) it is large enough to provide reliable estimations. at the moment, data provided by private entities represents the most reliable public source of information to explore the big picture from the perspective of people [11] . such sources of data include the ones provided by google, facebook and apple, as described in section 2. the main issue when working with private data sources is the imperfect knowledge regarding its nature. for privacy preserving reasons, raw data is never provided. instead, data goes through heavy pre-processing and anonymization procedures [12] . additionally, the conditions under which data is collected are not fully transparent either, as baselines and contextual information are often missing. to partially overcome these issues, in this work we investigate the relation between the different private data sources, and how can they be used complementary to provide a better understanding of mobility. for our analysis, we focus on the case of spain. the covid-19 pandemics in spain, and the political measures taken to control its spread in the country, provide an appropriate experimental setup. spain implemented a complete and sudden lockdown on march 15, 2020, while the first mild restrictions were put in place less than a week before (progressive closure of schools and banning of large gatherings) [5] . after the pandemic peaked in april, spain gradually recovered mobility and services over the span of two months (may and june). we will study spain's demographics and consider the relation between restriction policies, social behavior and pandemic evolution. this could be helpful for reacting to future mobility crisis. we will also study its progression towards a new normality in comparison with the old one. this could be helpful for the adaptation of mobility policies to the new social setting. the rest of the paper is structured as follows. first, we describe the data used in this work in §2. then we review the social context of the study in §3, including both the timeline of implemented policies, and an overview of spain demographics. most of our analysis takes place in §4. this includes a general study of mobility trends for all regions and data sources ( §4.1), a discussion on the anomalies observed ( §4.2), an analysis on the daily trends ( §4.3) and some insights on the new normality ( §4.4). finally, we review our conclusions in §5. for this work we have considered the data published by the facebook data for good program (fdg) [13] , by the google mobility assessments [14] , and by apple [15] . at the time of our analysis apple only provided one single mobility metric, the number of direction requests made from the apple maps application. in practice, this measure was too different from the other sources as to be directly compared. also, while it was possible to assess the anonymization effect on the data for fdg and google, we have not been able to do the same for apple. for these reasons, we disregard apple data in this analysis. private mobility indices are typically provided at a certain level of aggregated granularity. with administrative level 1 being country, level 2 in spain corresponds to region (17 autonomous communities or cc.aa.), and level 3 to province (50 of them in spain). as we will see next, the google data is provided at level 2 (regions), while facebook data is provided at level 3 (province), a smaller geographical granularity. this will be aggregated to level 2 on our work. facebook data for good presents multiple data sets of mobility, all of them with a general description of how they are obtained. there are two main data sets of particular relevance for the purpose of this work: a remain-in-tile index and movementbetween-tiles index [16] . both are based on gps data from a sample of users that have activated the tracking system on their mobile phones, dividing the space in level 16 tiles (squares of roughly 500 × 500 m). the first one, remain-in-tile, provides the percentage of people that remains in the same tile, computed as the total ratio of mobiles providing signal which do not change of tile during a whole day. the second one, movement between tiles, estimates mobility by computing how many different tiles are visited by the sample of people, compared with the same number during the same day of the week previous to the pandemics (february 2020) [12] . for the rest of this work, we will use the remainin-tile index, as it provides a more pure measurement. notice the remain-in-tile is an absolute measure (e.g., it does not depend on a baseline). this makes it straightforward to interpret, and a good candidate for counting people who are following confinement, as long as the population who has accepted to be pinned represents a good sample of the population. the anonymization of data prevents us from evaluating the fit between the data sample and the overall population distribution. a certain amount of bias is to be expected, as the penetration of smartphone and facebook varies significantly among cohorts. as with google, the elder population and the younger population could be slightly underrepresented, which may entail a certain bias. nonetheless, the size of the sample lead us to believe that the data can provide a reliable picture. we assess facebook's sample size in two ways. first, by considering facebook market penetration in the smartphone market, which is around 50-60% [17] with smartphones being available for 70% of the population. the subsample with an activated tracking system only needs to be around 10% to have a sample of 4% of the total population. even 1% of the population would be a very large sample in any kind of poll. second, data of active facebook users is also available from facebook geoinsights maps and indicates that 1% is indeed the typical order of magnitude of the sample. the fact that facebook requires a minimum of 300 active users to provide data (for anonymity reasons) allows us to validate the size of the sample. for example, the province of spain with the smallest population is soria, with roughly 90,000 people. this province and has never pined less than 300 people for the whole period under study (i.e., there is data for soria for all days). this means that the sample in soria is always above 0.5% of the population. we have no reason to suspect a lower coverage on the other provinces of spain. facebook data is provided at level 3 (province) granularity. in order to compare with google data (which is only available at level 2), we need to aggregate it. any level 2 region is a sum of one or more level 3 provinces. to compute the level 2 data from level 3 values we use the average of provinces weighted by their population. by doing so we obtain the ratio of people that remain in tile at level 2 aggregation for facebook. on april 2, 2020, google released its covid-19 community mobility reports [14] . since then, google periodically releases anonymyzed mobility data 1 , organized in a set of categories (e.g., retail, recreation, groceries, pharmacies, parks, transit stations). this data is always provided at administrative level 2 granularity (region). in addition to those, google also releases a residential and a workplace measure, estimating how much time is spent at those places. in the case of residential, it is based on the average number of hours spent at the place of residence for each user within a geographic location. this data is collected from those users opting in to location history, and is processed using the same algorithms used for the detection of user location in google places, offering an accuracy of around 100m in urban areas [18] . google mobility reports releases relative information for every day of the week. that is, mobility with respect to a baseline: the first five weeks of the year, from january 3 to february 6. given the large number of users of google maps there is no doubt that the sample size of google is also large compared with any standard polling sample. however, the fact that google does not provide an absolute value in the way facebook does presents researchers with serious challenges. the most important is that the baseline can be indeed corrupted by local festivities or changes in the normal baseline due to large-scale celebrations. as we will see, lack of access to the baseline prevents the direct comparisons of regions, and even days, and can complicate its interpretaion. google's residential index, which indicates the estimated time spent at the residence, should strongly correlate with facebook's remain-in-tile. i.e., a large fraction of people remaining at home should increase both indices. their direct comparison should provide context for their interpretation, and insights on their applicability. the first cases covid-19 in spain were reported on january 31st, in the canary islands. for roughly a month, all detected cases were imported from other countries. the first cases of community transmission (i.e., source of contagion unknown) were diagnosed on the february 26. in march is where the scope of our analysis begins. the number cases reached 999 on the 9th. at this point a few several regional governments took the first generalized measures, with the closure of schools. this escalated until march at administrative level 2, spain is composed by 17 regions and two autonomous cities (ceuta and melilla). the latter are not included in the analysis because their distinct nature would require a separate study. even so, the 17 regions of spain differ significantly in socio-economic factors, as shown in table 1 . two regions (catalonia and madrid) include the largest metropolitan areas, and had the most covid-19 cases in absolute terms. the rest of regions can be categorized as being dense or sparse (as determined by the population density), and as having their population centered on urban areas or not (as determined by the percentage of people living in cities with 50k or more inhabitants). cantabria and aragon are examples of the most uncommon cases, the first being fairly dense, with population centered on small towns, and the second being sparse, with population centered on cities. extremadura is a prototypical sparse and rural region, while madrid is extremely dense and urban. in our analysis, we focus on facebook's remain in tile index, and on google's residential index. the former is scaled (multiplied by 50) to approximate google's scale and facilitate the interpretation of figures. we have no means to assess the relation between the scale of both indices, which is why we avoid interpreting the relative volume between indices. nonetheless, as we will see in figure 1 , our scale provides a good approximation. our first analysis is focused on the general mobility trend of spanish regions during the peak of the pandemic. for this, we focus on the three months around the peak of the pandemic in spain, as shown in figure 1 . these are march, april and may. in this period, mobility in spain went through at least first of all, let us remark the lack of significant differences among regions with regards to the general trend. all show the same overall behavior through time. mild differences exist in the degree of confinement, and on the recovery speed during deconfinement. madrid for example reaches a higher level of confinement and recovers mobility much slower than extremadura. the main distinctive factor of regions comes from the occurrence of periodic peaks in the data (sometimes upwards, sometimes downwards). we discuss those in detail on the following section. according to both mobility indicators, the spanish society assumed and implemented the general lockdown in a matter of 48 hours (from march 12 to march 14) . this level of mobility restrain was sustained for seven weeks. for reference, wuhan, the source of the pandemic, held its lockdown for approximately eight weeks [19] . spain mobility was minimized from march 15, with the declaration of the state of alarm, to may 2, with the approval of de-confinement measures (consecutive orange, red and orange bands in figure 1 ). after may 2, restrictions were gradually lifted, causing a progressive recovery of mobility that officially ended on june 21. within the seven lockdown weeks, two were under hard-lockdown. since restrictions were enforced by police, mobility in this two week is a good estimate of the maximum mobility restriction that can be held in spain while keeping essential services running. to further understand the role of the hardlockdown, we compute an estimate of its impact on mobility. we are interested in its effect when compared to mobility under regular lockdown. the regular lockdown includes five weeks of lockdown data (the last two of march and the last three of april) during which traveling to industry and construction workplaces was allowed. this sort of lockdown is assumed to have a less damaging effect on the economy, but it enables infection among co-workers. to compare mobility between both periods we measure the corresponding area under the curve. the higher the area, the higher the constraint. for each period, we normalize the area by the number of weeks. the results for all 17 regions are shown in figure 2 as a separate distribution for the google and facebook indices. in this case, facebook is the most interesting source, since it is an absolute measure and allows us to measure volume of people. this data indicates an increase between 10% and 15% in the people who stayed put. the relevance of that number for the containment of the pandemic is unknown to us. i.e., we do not know which would have been the evolution of the pandemic if hard-lockdown had not been implemented. one may argue that many [20] shifted two weeks early to approximate contagion date. more people would have been infected, since regular lockdown enables transmissions on the workplace. however, the number of detected cases did not show any change after hard-lockdown ended, it continued to decrease at a similar rate. one may also argue that the hard-lockdown had a psychological effect on society, boosting resiliency to confinement. looking at how mobility recovers right after hardlockdown is lifted, this seems to be a valid hypothesis. if that were the case, a state of alarm without hard-lockdown may have lost adherence faster. to observe the effect and timing of the policies implemented, we can compare it with the status of the pandemic. for that purpose we use the number of daily reported cases, plotting it against the mobility curves. to approximate the date of the contagion from the date of report, we shift this data two weeks early. this is motivated by current estimates [21] , which assert that the vast majority of covid-19 patients develop symptoms (if any) before day 14 after contagion. figure 3 shows this comparison, for the case of madrid [20] , the region with the most cases and the strongest lockdown adherence. in figure 3 we observe the beginning of the lock-down overlapping with the initial containment of the pandemic. that is, the number of contagions halts its exponential trend around the date of the state of alarm declaration. although we do not know the exact role of the state of alarm, it is shown to be a clearly correlated factor. the seven weeks of general lockdown coincide with the seven weeks of strongest pandemic rate reduction. that is the time it took the pandemic to reach a basal situation in the region, with less than 100 reported cases a day. according to this estimate, this situation may have been reached around the starting date of the de-confinement process. if all this was the case, the duration of the spanish lockdown (7 weeks) was a very good fit to the evolution of the pandemic. a shorter lockdown may have induced a significantly higher risk of relapse, and a longer one seems unnecessary in sight of the contagion numbers estimated to be taking place in early may. let us remark that during the crisis policy makers could only use current daily cases for their decision making. that is, without the 2 week shift we performed in figure 3 . the de-confinement process was therefore a bolder (and riskier) initiative than figure 3 illustrates. the general trend described in the previous section is explained by the different stages of confinement enforced by the spanish government. on top of this trend, we can see the occurrence of a number of peak values, happening periodically and on all regions. these peaks occur mostly on sundays (marked with grey vertical lines in figure 1 ), and to a lesser degree on fridays and saturdays. let us discuss sundays and fridays in detail, since saturdays seem to be a middle ground, transitioning between both. sunday peaks are anticorrelated in terms of relative mobility (google goes down) and absolute mobility (facebook goes up) . this means the number of people moving is small when compared to near-by days of confinement. it also means that the number of people moving is not so different from what it used to be, when compared to the same day of the week on normality. in other words, people were not moving much on sundays before the covid-19 crisis, and during it they were moving less. accordingly, even though the decrease on mobility on sundays is not as big as on other days of the week, it still accounts for the day of the week with less absolute mobility. that would make sundays the best candidates for the mobility of risk population. in contrast, friday peaks exhibit a rather different behavior. in this case, the relative mobility decreases sharply (google goes up), while the absolute mobility remains stable or decreases mildly (facebook goes flat or slightly up). this indicates fridays are the days with the most different mobility patterns with respect to the previous normality (relative change), which speaks of the high mobility taking place on a normal friday. on these days is when society is showing its biggest change, leveling mobility to the rest of the working week. that would make fridays the best candidates for communication and support (e.g., mental health assessment). let us remind the reader that these insights may be reinforced by the bias in the data, which favours the presence of the young segments of society. let us now conduct an experiment to validate the hypothesis that peaks are related to the relative or absolute nature of measures. we transform the facebook measure from an absolute one to a relative one, using as a baseline analogous remain in tile data, from february 24 to march 8, before the first regional restriction measures. this baseline is computed weekday-wise, like google's. the result is shown in figure 4 , together with the original facebook data, and the google measure. the first obvious change are sundays, which now peak downwards like google. in fact, our relative facebook measure perfectly aligns with google around weekends (friday to monday) during the whole lockdown. this may be caused by differences in the data (both data sources have different resolutions to measure movement), or by differences in the baselines given the daily consistency. understanding the nature of these peaks is important because of the effect these may have on certain metrics. as shown in figure 5 , pearson correlation between facebook's remain in tile and google's residential index varies significantly from month to month. on march, mobility exhibited a very clear trend as a result of the establishment of confinement measures. in this setting, the correlation between both indices is clear (around 0.9 pearson correlation on average), and the peaks are not disruptive enough as to alter it. on the other hand, mobility during april was stable, as the whole month was under lockdown. this entails an overall flat behavior of the indicators. a context in which the inverted peaks have a dramatic effect, destroying all correlation between indices. finally, may appears as a middle ground. there is a generalized mobility trend, which reduces the upsetting effect figure 4 : evolution of mobility according to facebook absolute change (blue), facebook relative change (green) and google (purple). facebook relative change has been divided by 100 to match the scale of the other two indices. the vertical bands (green, orange, red, orange, green) correspond to pre-confinement, state of alarm declaration and lockdown, hard-lockdown, lockdown and de-confinement stages. grey vertical lines are aligned with every sunday. of the peaks, but the trend is not strong enough as to completely overpower the noise. week days have an important role in the characterization of mobility. let us now study the same data, but this time from the perspective of days. to do so we plot the facebook and google mobility indices as two different axis. figure 6 provides two visualizations for the first three months of the pandemic in spain. on the top row, the color gradient shows the change through time, week by week. on the bottom row, week days are color coded to illustrate the differences between days. in these plots, horizontal axis show absolute change (the more to the right, the bigger number of people stay at home) while the vertical axis shows relative change (the more up, the more percentage of people stay at home with respect with normal instances of that day). the first visible thing in figure 6 is the correlation between both values, as all data is mostly gathered around the diagonal. the top row shows the evolution of mobility, starting from the axis origin (bottom left) and suddenly jumping to the top right quarter of the plot as lockdown is implemented. the last friday and saturday before the lockdown (second week of march) are the only days in the middle of that jump. during confinement (april) data is rather stable in that area, until the de-confinement measures (may) bring it down and left again, but this time in a slow manner. the visualization using both google and facebook as axis shows the clear correlation be-tween them. in general, as relative mobility increases/decreases, so does absolute mobility. however, this relation seems to be somewhat dependant on the day of the week. as shown on the bottom row of figure 6 , sundays have a rather different behavior in terms of relative mobility (it shows less affection in this metric) while friday represents the opposite (it shows more affection in relative mobility). this is a different visualization of the same phenomenon observed in the peaks of figure 1 and discussed in §4.2. on the second half of may, spain started to lift the confinement measures that had been in place in the country for two months [5] . the process was asymmetrical, with regions with better pandemic indicators (i.e., number of daily cases, number of available hospital beds, etc.) de-confining faster than others. detailed maps of the differential treatment of regions can be found in governmental sources [22] this process ended on june 21, when the state of alarm (and all mobility restrictions) was lifted. on that date, the whole country officially entered the new normality. figure 7 includes the last four weeks of state of alarm (but without a generalized lockdown), and the first one of new normality. to facilitate visualization we change zoom in and the axis scale with respect to figure 6 . nonetheless, to enable comparison with the rest of the period under study, in figure 8 we plot the same data using the scale used in figure 6 the progression of mobility towards the axes origin is still visible and this smaller scale as weeks go by (in color gradient), for both working days (monday to friday) and weekend days (saturday and sunday). to compare the new normality with the old one, we must focus on the google axis, since this is relative to a baseline (january 3 to february 6). on the weekends mobility is already at google baseline levels, with all values between -5 and 5 on the last week (the new normal one). in contrast, working days are showing a higher difference with respect to the baseline, with several values in the last week between 5 and 15 in the google axis. this indicates that the change implemented by the spanish society during the new normality is focused on working days, while weekends are back to how they were. the recovery of old normality is not homogeneous amount regions either. catalonia and madrid, the regions with the biggest metropolitan areas, clearly lag behind. asturias, navarre, la rioja, region of murcia, extremadura and galicia are way ahead. of those, only murcia has a population density above 100 (132 people/km 2 ), hinting a potential relation with this indicator. the lag of catalonia and madrid during the first 4 weeks is likely related with the fact that these regions were slightly behind in the removal of restriction measures. however, during the 5th and last week of data all regions were under the same conditions, and catalonia and madrid still exhibit higher levels of mobility reduction. this may be related with the role of large metropolitan areas, were it is harder to keep a safe distance, and with the fact that both regions reported the highfigure 6 . the data of this figure is zoomed in and split between working days and weekend days in figure 7 . est absolute volume of infections during the pandemic. both these factors are strong psychological enablers of self-responsibility, which may have an effect of adherence to mobility reduction during the new normality. in this work we consider the use of private data sources (google and facebook) for assessing the levels of mobility in a country like spain. by doing so, we draw conclusions on two fronts. first, on the behavior and particularities of private data sources. and second, on how mobility changed during the covid-19 pandemic in spain. regarding private data sources, we have shown the differences between using an absolute measure (like facebook) and a relative measure (like google). both of them have limitations when used in isolation. the former lacks a contextualization of its values, while the latter depends entirely on the baseline used. when used together, they provide a visualizing of mobility where consistent patterns can be easily identified (as presented later in this section). for specific purposes, using a single data source may suffice, as long as it fits the goal: • an absolute measure like facebook's can be very useful for epidemiologic purposes, as it provides an pure measurement of mobility. that includes estimating number of contacts in a society, modeling the spread of the virus, and measuring the impact of policies on absolute mobility. • a relative measure like google's can be very useful for socio-economic purposes, as it provides a contextualized measurement of mobility. that includes understanding the change caused by the new normality, and the economic impact of mobility restriction policies. on the second topic of this paper, the analysis of spanish mobility during the covid-19 pandemic, we extract several conclusions. on one hand, data shows a huge mobility containment, sustained for a month and a half (march 15 to may 1st, approximately), very close to its theoretical limit (as represented by mobility during the hard-lockdown). this duration was sufficient to contain the spread of the virus and bring infection numbers down to traceable scale. in hindsight, the policies implemented in spain seem appropriate and proportional to the severity of the situation. that being said, the role, timing and convenience of the hard-lockdown remains to be further discussed. our work shows a relatively modest contribution of this policy to mobility reduction. on the other hand, the hardlockdown may have had an effect on prolonging adherence. our work identifies mild differences between regions during the three months of restricted movement. certain regions had a stronger adherence to confinement than others, mostly in relative terms. this may be caused by regional differences in prepandemic mobility, which is used as baseline for the relative measurement. a similar artifact are the inverted peaks of weekends, where a relative measure spikes down and an absolute measure spikes up. as demonstrated, this the result of combining a measure relative to the weekday with an absolute measure. we also saw significant differences among days. weekends exhibit the highest volume of mobility reduction in absolute terms, even during the hardlockdown, when traveling to work was forbidden for all except essential services. at the same time, weekends have the smallest mobility reduction in relative terms, indicating that the effort society had to make in this regard with respect to its previous patterns was smaller. fridays and sundays are particularly relevant days, the first because it represents the biggest change from normal behavior, the second because it represents the biggest absolute decrease in mobility. these particularities could be exploited for the general good. finally, we analyzed the new normality by looking at the weeks of de-confinement, up until june 27, a week after the state of alarm was lifted on the whole of spain. in this period, we found saturdays and sundays to be already at pre-pandemic levels of mobility. in contrast, working days (monday to friday) still show significant differences. the new normality also shows differences between regions, particularly for working days. regions with large metropolitan areas exhibit a reduction in mobility between 4% and 14% after restrictions were lifted. indeed, the new normality is most new on urban working days. economic and social consequences of human mobility restrictions under covid-19 spread and dynamics of the covid-19 epidemic in italy: effects of emergency containment measures presence and mobility of the population during covid-19 outbreak and lockdown in italy the effect of human mobility and control measures on the covid-19 epidemic in china ecml covid dashboard the spread of awareness and its impact on epidemic outbreaks evolución del nivel de movilidad del conjunto de provincias openflights rolls out new maps, moves to github mobile phone data for informing public health actions across the covid-19 pandemic life cycle quantifying international human mobility patterns using facebook network data aggregated mobility data could help fight covid-19 protecting privacy in facebook mobility data during the covid-19 response facebook data for good public datasets wilson mobility trends reports facebook disaster maps: aggregate insights for crisis response & recovery estudio anual mobile en españa 2019 y tendencias google timeline accuracy assessment and error prediction china scrambles to curb rise in imported coronavirus cases, wuhan eases lockdown situación y evolución de la pandemia de covid-19 en españa the incubation period of coronavirus disease 2019 (covid-19) from publicly reported confirmed cases: estimation and application plan para la transición hacia una nueva normalidad we would like to thank facebook and google for releasing the data that made this work possible. we also appreciate the insights and support of amaç herdagdelen, alex pompe and alex dow on the interpretation of peaks. part of this work was done under the global data science project for covid-19. we would also like to thank daniel lópez-codina, sergio alonso, and clara prats for fruitful discussions. finally, part of this research has received funding from the european union's horizon 2020 programme under the sobigdata++ project, grant agreement num. 871042. key: cord-348269-6z0kiapa authors: nguyen, quynh c.; huang, yuru; kumar, abhinav; duan, haoshu; keralis, jessica m.; dwivedi, pallavi; meng, hsien-wen; brunisholz, kimberly d.; jay, jonathan; javanmardi, mehran; tasdizen, tolga title: using 164 million google street view images to derive built environment predictors of covid-19 cases date: 2020-09-01 journal: int j environ res public health doi: 10.3390/ijerph17176359 sha: doc_id: 348269 cord_uid: 6z0kiapa the spread of covid-19 is not evenly distributed. neighborhood environments may structure risks and resources that produce covid-19 disparities. neighborhood built environments that allow greater flow of people into an area or impede social distancing practices may increase residents’ risk for contracting the virus. we leveraged google street view (gsv) images and computer vision to detect built environment features (presence of a crosswalk, non-single family home, single-lane roads, dilapidated building and visible wires). we utilized poisson regression models to determine associations of built environment characteristics with covid-19 cases. indicators of mixed land use (non-single family home), walkability (sidewalks), and physical disorder (dilapidated buildings and visible wires) were connected with higher covid-19 cases. indicators of lower urban development (single lane roads and green streets) were connected with fewer covid-19 cases. percent black and percent with less than a high school education were associated with more covid-19 cases. our findings suggest that built environment characteristics can help characterize community-level covid-19 risk. sociodemographic disparities also highlight differential covid-19 risk across groups of people. computer vision and big data image sources make national studies of built environment effects on covid-19 risk possible, to inform local area decision-making. the covid-19 pandemic has caused approximately 150,000 deaths in the united states as of 29 july 2020 [1], and has had unprecedented negative effects on the u.s. economy and households in numerous ways. the unemployment rate rose up to 14.9% in april and the gdp fell by 1.2% in the first quarter in 2020, which is the largest decline since the great recession [2, 3] . yet the negative impacts of covid-19 are not evenly distributed. about half of lower-income u.s. households lost employment income. about 62% of hispanics and 57% of black adults were in households that experienced employment income loss compared to 45% of whites [4] . moreover, the spread of covid-19 is not evenly distributed. racial/ethnic disparities in covid-19 infection and mortality are coming to light, with disproportionate numbers of covid-19 cases and deaths among racial/ethnic minorities compared to non-hispanic whites [5, 6] . some of these differences reflect the living and social conditions faced by racial/ethnic minorities. for instance, institutional racism that produced residential segregation may increase the likelihood that racial/ethnic minorities live in densely populated areas with substandard and crowded housing conditions impede social distancing [7, 8] . a recent analysis suggested that counties that are predominately black have three times the infection rate of covid-19 compared to white majority counties [9, 10] . covid-19 can spread through droplets that are released when people talk, cough or sneeze or when people touch a contaminated surface and then touch their nose or mouth [11] . research has identified a myriad of important factors that influence covid-19 transmission including anti-contagion governmental policies [12] , community adherence to preventative health behaviors (e.g., mask wearing, social distancing) [13] and other environment characteristics like air pollution. emerging research has found higher levels of air pollution may increase covid infection rates as well as covid-related mortality, possibly because particulate matter can act as a carrier of the virus and also compromise the baseline health of communities that have chronic exposure to air pollution [14] . in the current study, we focus on a neglected area of research, the potential relationship between built environment characteristics and covid-19 cases. to conduct this investigation, we utilized the largest collection of google street view images that has been leveraged for public health research to characterize neighborhood environments. in examining associations between built environment characteristics and covid cases, we controlled for demographic compositional characteristics of areas and population density, which has previously been utilized in econometric studies as a proxy for air pollution and other factors found with greater prevalence in urban areas [15, 16] . neighborhood environments may structure risks and resources [17] that produce covid-19 disparities through several pathways. firstly, neighborhood built environments that allow greater flow of people into an area or impede social distancing practices may increase residents' risk for contracting the virus. a recent study that used data from pregnant women in new york city revealed that overcrowding housing units have higher chances of contracting covid-19 [18] . neighborhoods with a mixture of residential and commercial uses (e.g., high prevalence of grocery stores and businesses), multiple lanes of traffic, and higher density of sidewalks, may allow more people to congregate in an area and more easily spread covid-19. additionally, previous studies found that physical disorder in the neighborhood environments is significantly associated with higher prevalence of chronic diseases [19] and poor self-rated health [20] , which also increases the chances of contracting covid-19 [21, 22] . physical disorder refers to features of the environment that signal decay, disrepair, and uncleanliness. examples of neighborhood indicators of physical disorder include vacant or abandoned housing, vandalized and run-down buildings, abandoned cars, graffiti, and litter [23] . physical disorder is often interpreted as an indicator of low neighborhood quality [24] . physical disorder is hypothesized to indicate a breakdown of social disorder and control, which reduces individual well-being and increases fear, mistrust, isolation, anger, anxiety, and demoralization [25] . mechanisms proposed include the daily stress imposed by environments that are deemed unsafe. previous research has connected physical disorder with an array of detrimental health outcomes including worse mental health, higher substance use, physical functioning and chronic conditions [26] . physical disorder might also indicate fewer resources for infrastructure maintenance and investment. communities with poor-quality housing stock may have less healthy indoor conditions, with consequences for baseline respiratory health. in this study, dilapidated buildings and visible utility wires overhead were utilized as indicators of disorder. visible utility wires hanging overhead are visually striking and may impact residents' aesthetic sense of their environment, altering perceptions of safety or pleasurability and influencing both mental health (by affecting stress levels) and physical health (by disincentivizing walking). other studies that have examined this indicator have been done outside the u.s., where they may also represent an unsightly presence and electrocution/electrical fire risk [27] . computer vision models have struggled with small objects, precluding us from labeling other indicators of physical disorder such as litter or trash [19] . investigations into neighborhood conditions are typically conducted on small scales for only certain cities or neighborhoods [28, 29] . when conducted, neighborhood data collection is expensive and time consuming, and then only available for certain time periods. currently, detailed neighborhood data come from neighborhood surveys, administrative data such as census data, and systematic inventories of neighborhood features. subjective assessments of neighborhoods from community residents can help identify factors that residents believe are most important to their health and increase understanding on how individuals differentially use and interact with their environment. however, self-reported neighborhood data can be influenced by participants' health status and cognitive function, resulting in "single source bias" [30] . the other neighborhood data we do have is mainly data on demographics (e.g., percent black). to our knowledge, our study is the largest to date using zip code level cases from 20 states to investigate associations between built environments and covid-19 cases. previous studies examining the distribution of covid-19 cases are only focused on one or two states [31] [32] [33] or larger geographies like counties [34] . google street view (gsv) images represent a massive, publicly available data resource that has high potential but is very underutilized for health research. it can be used to extract information on physical features of the environment at point locations all over the country. consistently constructed neighborhood quality indicators across large areas are severely lacking. while some studies have used human coders to classify environmental features seen in google street view images [35] this approach is not feasible on the massive scale necessary to compare thousands of u.s. neighborhoods. the development of data algorithms that can automatically analyze big data sources such as street view images will create a new national data resource for timely decision-making to mitigate the impact of covid-19 and future outbreaks on health and health disparities. the purpose of characterizing built environments that have higher covid-19 risk is to identify places where additional safeguards and resources are needed. study aims and hypotheses. in this study, we investigated how the built environments affect covid-19 cases at the zip code level. we utilized 170 million gsv images sampled at 50 meters apart and computer vision models to comprehensively characterize neighborhood conditions across the united states. from gsv images, we created indicators of urban development (non-single family home, single lane roads), walkability (crosswalks, sidewalks), and physical disorder (dilapidated building, visible utility wires). we hypothesize that built environments characterized by greater urban development, walkability, and physical disorder will have higher covid-19 infection rate. street view image data collection. we utilized google street view's application programming interface (api) to capture street view images of our search set. image resolution was 640 × 640 pixels. we surveyed all u.s. roads and obtained 4 images from each sample location with angle views at 0, 90, 180, and 270 degrees, thus permitting fuller capture of the surrounding area of a point location. in total, 164 million images were obtained in november 2019. image data processing. convolutional neural networks (convnets) [36] [37] [38] achieve state-of-the-art accuracy for several computer vision tasks including but not limited to object recognition, object detection, and scene labeling. for example, the state-of-the-art accuracy of imagenet [39] with 1000 categories and over one million image samples is improved every year using convnet-based methods. the imagenet dataset contains images from various categories (e.g., "moped", "granny smith apple") and corresponding category labels. models trained on this dataset use trial and error to learn combinations of colors, shapes, and textures that are relevant to a wide variety of image interpretation tasks, and therefore can be used as a starting point for creating computer vision models for tasks where labeled training data is scarce. a convnet model "pre-trained" on imagenet can be "fine-tuned" using a smaller amount of training data from the desired task, which delivers strong classification performance without requiring the vast training data and computational resources necessary to train the original convnet. neighborhood definitions. zip codes were utilized as neighborhood boundaries because various health departments across the country are releasing covid-19 cases by zip code. to arrive at the neighborhood indicators, we processed street imagery and then combined information on all street imagery within a zip code to arrive at zip code-level summaries (e.g., the percentage of images in a zip code that contain a sidewalk). built environment indicators. to create a training dataset for our computer vision models, from december 2016-february 2017, we manually annotated 18,700 images (from chicago, illinois; salt lake city, ut; charleston, west virginia; and a national sample). these locations were chosen to capture heterogeneity in neighborhood environments across geographically and visually distinct places with varying population densities, urban development, and demographics. labelers included the principal investigator and three graduate research assistants. inter-rater agreement was above 85% for all neighborhood indicators. each image received labels for these binary neighborhood characteristics: (1) street greenness (trees and landscaping comprised at least 30% of the image-yes/no), (2) presence of a crosswalk, (3) single lane road, (4) building type (single-family detached house vs. other), and (5) visible utility wires. green streets were utilized to indicate lower urban development. single lane/residential roads limit the number of cars and hence flow of people. non-single family home was utilized as an indicator of residential and commercial mixture. crosswalks were utilized as an indicator of walkability. visible utility wires were utilized as indicators of physical disorder. we randomly divided the dataset into a training set, a validation set, and a test set. the training and validation set contained 80% of total labeled images and the remaining 20% was used as a test set to evaluate the model's performance. once the hyper-parameters were chosen, each model architecture was trained multiple times. note that neural network training is stochastic even when starting from the same initialization and using the same training set, therefore, multiple training runs are used to assess the mean and standard deviation of the error. the testing set remained unobserved until the best models had been selected using the training set. we assessed the final quality of the model using the test set. we first resized all the images to the size 224 × 224 for processing. we then trained a standard deep convolutional neural network architecture-visual geometry group vgg-19 [36] in tensorflow [40] with sigmoid cross entropy with logits as the loss function. the weights of the network were initialized from imagenet weights. adam optimizer was used with batch size 20. training took 20 epochs and started with learning rate 10 −4 . we considered the model saved in the last epoch as our final model. accuracy of the recognition tasks (agreement between manually labeled images and computer vision predictions) were the following: street greenness (88.70%), presence of crosswalks (97.20%), non-single family home (82.35%), single lane roads (88.41%), and visible utility wires (83.00%). these figures were consistent with a separate, semi-supervised learning approach. below, we describe the model building process for two additional neighborhood indicators that utilized different training datasets. dilapidated building indicator. our training dataset consists of approximately 29,400 google street view images captured from baltimore and detroit based upon administrative lists from city governments on vacant buildings and buildings marked for demolition from 2014-2018. we randomly split this dataset in the ratio 80:20 for validation to obtain about 23,500 images for training and 5900 for validation. the dataset has an equal number of normal and dilapidated buildings. we then trained a standard deep convolutional neural network architecture-resnet-18 [38] in pytorch [41] with nll loss as the loss function. for the dilapidated building indicator, the resnet-18 model produced an accuracy of 89.1% and a f1 score of 89.1. sidewalk indicator. our training dataset consists of about 24,316 images captured from google street view from new jersey that had been manually labeled. we randomly split this dataset in the ratio 80:20 for validation to obtain 19,452 images for training and 4864 for validation. the minority label images were oversampled so that the dataset has an equal number of sidewalk present and absent cases. we then trained a standard deep convolutional neural network architecture-resnet-18 [38] in pytorch [41] with nll loss as the loss function. for the sidewalk indicator, the resnet-18 model produced an accuracy of 84.5% and a f1 score of 81.0. covid-19 cases. to our knowledge, there is no national data source for zip code covid-19 cases, with the centers of disease control and prevention and john hopkins covid-19 map only showing county level cases as the lowest level of geography. to obtain zip code covid-19 cases, we visited state and county health departments that had covid-19 information (31 websites in total; 12 websites utilize arcgis dashboards, and 19 utilized a mixture of pdfs, csv files, and tableau/powerbi embedded websites). data were obtained from official government websites and actively maintained github repositories using various methods. this collection process was automated using python packages including scrapy, selenium, beautifulsoup, and requests. specifically, for websites with arcgis map layer, we used arcgis query services to query the feature layer; for websites with csv data files to download, we automated the download process from the websites; for static website tables, we leveraged scrapy or beautifulsoup packages to harvest the web content; for websites with pdf files, we first downloaded the pdf files and utilized ocr technology to convert the data into the csv format. some states have report data for all zip codes, but others only report for certain cities or counties. zip code confirmed covid-19 cumulative cases as of 21 ) . covid-19 cases varied across zip codes with some zip codes reporting zero or few cases and others reporting hundreds of cases. about 50% of zip codes had 15 or fewer cases ("cold spots") and 10% had 250 or more cases ("hot spots"). in this study, we investigated whether zip code built environments can help explain some of the variation in covid-19 cases across 20 states. for each zip code, we calculated the percentage of total number of images that contained a given built environment indicator (e.g., number of images with a sidewalk/total number of images) *100 = percent with sidewalk. from there, we created tertiles and classified each zip code based on their percentage, with the lowest tertile as the reference group. we fit poisson regression models to estimate associations between gsv-derived built environment characteristics and covid-19 cases, controlling for potential confounding variables. log of total population at risk was used as the offset variable, to account for varying population sizes across zip codes. goodness-of-fit chi-square tests indicated the data fit with the poisson model form. all predictor variables were standardized with a mean of 0 and a standard deviation of 1. coefficients from poisson regression models were exponentiated to arrive at estimates of incidence rate ratios for a one-unit change in the predictor variable (i.e., one standard deviation change). separate regressions were run for each built environment indicator given moderate associations between the built environment indicators that varied from −0.23 for single lane roads and visible wires to −0.83 for green streets and non-single-family homes. models controlled for population density, household size, median age, household income, poverty rate, unemployment, percent with less than a high school education, percent asian, percent black, and percent hispanic. covariate information was obtained from the american community survey 2018 5-year estimates, with the exception of population density and household size which were obtained from the 2010 us census. we hypothesized that zip codes with more crosswalks and sidewalks (indicators of walkability), non-single family homes (an indicator of mixed commercial/residential uses), more visible wires and dilapidated buildings (indicators of physical disorder) would be associated with more covid-19 cases. we hypothesized that zip codes with more single lane roads (an indicator of lower urban development) would be associated with fewer covid-19 cases. stata ic15 (statacorp lp, college station, tx, usa) were used for all data analyses. this study was approved by the university of maryland institutional review board. figure 1 presents examples of processed google street view images. predictions were algorithm-derived labels for neighborhood features. "true" labels were manual annotations provided by the research team. our computer vision model was able to classify even winter scenes as "green streets" because the model was trained with manually annotated images to recognize tree branches as landscaping. station, tx, usa) were used for all data analyses. this study was approved by the university of maryland institutional review board. figure 1 presents examples of processed google street view images. predictions were algorithm-derived labels for neighborhood features. "true" labels were manual annotations provided by the research team. our computer vision model was able to classify even winter scenes as "green streets" because the model was trained with manually annotated images to recognize tree branches as landscaping. table 1 displays descriptive statistics at the zip code level. on average, approximately 25% of images in a zip code contained a building that was not a single family home, 20% of images had a sidewalk, 2% with a crosswalk, and 44% with visible utility wires. dilapidated buildings had a prevalence of 18%, while single lane roads (65%) and green streets were more prominent (87%) ( table 1) . we examined covid-19 cases in 8171 zip codes across 20 states in the united states with an average of around 546 cases per 100,000. table 2 presents the results of our poisson regression analyses examining the relationship between gsv-derived built environment characteristics and covid-19 cases. we found that zip codes with a standard deviation increase in sidewalks had 40% more cases (table 2) . a standard table 1 displays descriptive statistics at the zip code level. on average, approximately 25% of images in a zip code contained a building that was not a single family home, 20% of images had a sidewalk, 2% with a crosswalk, and 44% with visible utility wires. dilapidated buildings had a prevalence of 18%, while single lane roads (65%) and green streets were more prominent (87%) ( table 1) . we examined covid-19 cases in 8171 zip codes across 20 states in the united states with an average of around 546 cases per 100,000. table 2 presents the results of our poisson regression analyses examining the relationship between gsv-derived built environment characteristics and covid-19 cases. we found that zip codes with a standard deviation increase in sidewalks had 40% more cases (table 2) . a standard deviation increase in crosswalks and non-single family homes was associated with 14% and 21% more cases, respectively. we also found that indicators of physical disorder such as dilapidated buildings or visible utility wires were associated with more cases. alternatively, single lane/residential roads and green streets were associated with fewer cases. zip codes with a standard deviation increase in single lane roads and green landscaping had 10% and 4% relative fewer covid-19 cases, respectively. additionally, population characteristics associated with increased coronavirus cases included household size, percent with less than a high school education, percent racial/ethnic minorities (in particular percent black), and population density. estimates for covariates varied because the gsv-derived variable was different in each of the models. correlations between covariates and the particular gsv-derived characteristic differed and hence the coefficient estimates for covariates also differed. nonetheless, the variation in estimates for covariates was generally small/moderate. across models, a standard deviation increase in percent with less than a high school education was associated with 42-54% increase in covid-19 cases. across models, percent black was associated with 17-29% increases in coronavirus cases. a standard deviation in population density was associated with 1-4% more coronavirus cases. mobility changes during the covid-19 pandemic may have increased the importance of neighborhood environments. google's community mobility report [42] indicates that out of six categories of movement (retail and recreation, grocery and pharmacy, parks, transit stations, workplaces, and residential), movement volumes declined in all categories except residential and parks ( figure s1 ). consequently, the neighborhood environment is crucial for containing the spread of coronavirus, as more residents may have limited activities to their immediate neighborhood surroundings. our study finds that neighborhood built environment may influence the spread and containment of covid-19. leveraging google street view images, we found that single-lane/residential roads and green streets were associated with fewer cases, while non-single family homes, sidewalks, and physical disorder were associated with more cases in the neighborhood. in other words, covid-19 risk is highest in more built-up, more walkable, and more physically deteriorated zip codes, and lower in zip codes with smaller, greener streets. these associations persist after controlling for urbanicity and sociodemographic indicators, suggesting a meaningful role for the built environment in influencing covid-19 risk. the study is one of the first to investigate the effect of neighborhood built environment on the spread of coronavirus at the zip code level. single-lane/residential roads and green streets are indicators of lower urban development and lower social contacts. green streets are especially prevalent in rural areas and suburban areas. conversely, neighborhood environment indicators such as non-single family homes, sidewalk presence, and physical disorder may facilitate the spread of coronavirus. the ability to perform social distancing is not equally distributed across neighborhoods, and it is more difficult to achieve in highly developed urban areas. one study found that it is impossible to implement effective social distancing in urban areas with homes in close proximity to each other, such as cape town [43] . the same argument can also be applied to densely populated areas such as new york city, which was the epicenter of the covid-19 pandemic in the u.s. residential settings other than single-family homes-for instance, apartment complexes-are more likely to be the source of infectious disease outbreaks. in 2003, the sars outbreak started in a 33-floor apartment block in hong kong [44] . shared elevators and shared space are both risk factors for covid-19 infection. sidewalks, on the other hand, are likely associated with more walking, and the majority of neighborhood sidewalks do not allow pedestrians to maintain the cdc-recommended 6-foot distance. in this study, we find that indicators of physical disorder (dilapidated buildings and unsightly visible utility wires) were connected with more covid-19 cases, possibly due to worse health and higher comorbidities that increase in disorderly neighborhoods. our study is significant because it strives to identify and make available novel indicators of neighborhood quality by leveraging big data resources and furthering the application of computer vision. we utilized google street view images as a time-and cost-efficient data source for the characterization of built environments involving close to 170 million images sampled 50 m apart. the inclusion of 20 different states with varying built environments and covid-19 burden further strengthened our study. our study found that neighborhoods with greater urban development, higher walkability, and physical disorder had higher coronavirus cases. nonetheless, our study also has limitations. the cross-sectional study design inhibits causal inference. although we have observed strong associations between neighborhood built environment indicators and coronavirus prevalence, we cannot conclude that these characteristics cause higher covid-19 rates. additionally, we were not able to control for local covid-19 resources (e.g., testing availability). however, we controlled zip code sociodemographic characteristics such as racial/ethnic composition and median income that are correlated with greater resource access. fine particle air monitor data from the u.s. environmental protection agency (epa) are not available at the zip code level(https://www.epa.gov/outdoor-air-quality-data/pm25-continuous-monitor-comparabilityassessments) and hence, we were unable to account for this characteristic in our analyses. air pollution can vary between areas and has been related to a variety of acute and chronic conditions [45, 46] , which can compromise health and place individuals at greater risk for more severe covid-19 illness. lastly, the study was u.s.-based; built environments, demographics, health policies, and other considerations vary across international settings and thus our study results might not generalize to other countries. nonetheless, gsv images have been utilized in international settings to examine neighborhood features, and thus has the potential to enable other countries to examine the influence of built environment characteristics on health and other outcomes [47, 48] . like other modes of data collection, image data can only capture a subset of features of a community. images do not capture all of the features of the neighborhood environment that may impact health outcomes. for instance, we were unable to capture indicators of perceived safety that impact people's willingness to walk in an area. additionally, google street view api provides the most recent image available for a location. however, areas differ with regard to the rate at which their gsv image are updated. in our dataset, image dates ranged from 2007-2019 and the median year was 2015. thus, the neighborhood data for certain areas might not reflect current conditions. moreover, rural areas tend to have older gsv images than urban areas, which may lead to differential measurement bias. in addition, not all types of built environment characteristics lend themselves to easy extraction by computer vision algorithms. objects that are small (e.g., litter), vary in appearance (e.g., dilapidated buildings), or are very rare in the dataset (e.g., graffiti) are difficult for computer vision models to predict with high accuracy. subjective characteristics such as the aesthetics or the visual appeal of an area are also difficult to model with computer vision. for subjective characteristics, use of crowdsource techniques that incorporate ratings from residents and visitors might be an effective way to create area-level ratings that capture the variability in these perceptions. besides the type of neighborhood features that can easily lend themselves to automatic extraction via computer vision models, the depth of neighborhood features that can be extracted may be limited. well-known neighborhood audit instruments such as the irvine-minnesota inventory [49] and the pedestrian environment data scan [50] can involve hundreds of different features. building a computer vision model to accurately extract each of these hundreds of features would be a difficult task. additionally, computer vision models using supervised learning approaches often require large training datasets composed of potentially tens of thousands of manually labeled images to adequately train models and hence investigative teams need to build in time and resources to create these large training datasets. in our study, to create our training dataset, team members took two months to label over 18,000 images for neighborhood characteristics. we also utilized administrative datasets that contained the locations of vacant building and buildings marked for demolition to provide enough training examples for our dilapidated building indicator. the use of computer vision and gsv images enables large studies of neighborhood features across broad geographies. however, the use of these automated technologies might limit the type, variety, and level of detail in neighborhood features that can be examined. for investigators interested in neighborhood characteristics for small areas, manual neighborhood inventories might be the appropriate choice to provide the necessary data. while computer vision is not without its limitations, using computer vision and millions of gsv images was the only feasible way to examine fine area-level built environment characteristics across 20 different states. gsv is a growing new area of research that has immense potential to shed light on the potential influence of neighborhood environments on a variety of health outcomes. the contextual factors that influence the spread of the coronavirus risk are poorly understood. with recent advances in computer vision and the emergence of massive sources of image data, we developed a data collection strategy utilizing geographic information systems to assemble a national collection of google street view images of all road intersections and street segments in the united states. we utilized this data bank and leveraged computer vision algorithms to produce neighborhood summaries of conditions that are linked with covid-19 risk through increased opportunity for person-to-person transmission. we found that indicators like greater urban development (mixture of residential and commercial buildings, multiple lanes of traffic), walkability (which may increase contact), and greater physical disorder were related to more coronavirus cases. our study results can help inform population-based strategies to mitigate covid-19 risk. a higher level of caution can be recommended for the reopening of communities with a heightened level of risk due to their neighborhood design. supplementary materials: the following are available online at http://www.mdpi.com/1660-4601/17/17/6359/s1, figure s1 : trends of six movement categories using google mobility report data, 15 february to 12 june 2020, united states. for disease control and prevention (cdc). cases in the u.s. available online the us labor market during the beginning of the pandemic recession congressional research service covid-19 pandemic's impact on household employment and income hospitalization rates and characteristics of patients hospitalized with laboratory-confirmed coronavirus disease 2019-covid-net, 14 states estimated non-fatal hospitalized cases, and patients known to have died 100,000 by race/ethnicity group as of 16 assessing capacity to social distance and neighborhood-level health disparities during the covid-19 pandemic the time of covid-19 what we could learn from the coronavirus outbreak on aircraft carrier theodore roosevelt covid-19 and african americans center for disease control and prevention (cdc) how the covid-19 spreads the effect of large-scale anti-contagion policies on the covid-19 pandemic the role of community-wide wearing of face mask for control of coronavirus disease 2019 (covid-19) epidemic due to sars-cov-2 covid-19 and air pollution: a dangerous association? population density and urban air quality. cesifo work. pap is there a link between air pollution and impaired memory? evidence on 34,000 english citizens the impact of the built environment on health behaviours and disease transmission in social systems associations between built environment, neighborhood socioeconomic status, and sars-cov-2 infection among pregnant women health and the built environment in united states cities: measuring associations using google street view-derived indicators of the built environment neighborhood disadvantage, disorder, and health prevalence and impact of diabetes among people infected with sars-cov-2 prevalence and impact of cardiovascular metabolic diseases on covid-19 in china systematic social observations of public spaces: a new look at neighborhood disorder evolutionary theory and neighborhood quality: a multilevel selection-inspired approach to studying urban property conditions disorder and decay: the concept and measurement of perceived neighborhood disorder. urban aff. rev an examination of processes linking perceived neighborhood disorder and obesity a local view of informal urban environments: a mobile phone-based neighborhood audit of street-level factors in a brazilian informal community baltimore neighborhood indicators alliance-the jacob france institute vital signs 11 reports area-aggregated assessments of perceived environmental attributes may overcome single-source bias in studies of green environments and health: results from a cross-sectional survey in southern sweden covid-19 and the unequal surge in mortality rates in massachusetts, by city/town and zip code measures of poverty, household crowding, race/ethnicity, and racialized economic segregation the differential impact of covid-19 across demographic groups: evidence from nyc. ssrn 2020 the determinants of the differential exposure to covid-19 in new york city and their evolution over time us county-level characteristics to inform equitable covid-19 response expanding tools for investigating neighborhood indicators of drug use and violence: validation of the nifety for virtual street observation very deep convolutional networks for large-scale image recognition imagenet classification with deep convolutional neural networks deep residual learning for image recognition imagenet large scale visual recognition challenge tensorflow: a system for large-scale machine learning pytorch: an imperative style, high-performance deep learning library google 2020-06-12 us mobility report novel coronavirus in cape town informal settlements: feasibility of using informal dwelling outlines to identify high risk areas for covid-19 transmission from a social distancing perspective apartment complex holds clues to pandemic potential of sars particulate air pollution and acute health effects chronic disease prevalence in women and air pollution-a 30-year longitudinal cohort study validation of a google street view-based neighborhood disorder observational scale exploring places of street drug dealing in a downtown area in brazil: an analysis of the reliability of google street view in international criminological research the irvine-minnesota inventory to measure built environments: development © 2020 by the authors. licensee mdpi we thank dina huang for her research guidance. the authors declare no conflict of interest. the funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results. key: cord-120442-qfgoue67 authors: zaman, anis; zhang, boyu; hoque, ehsan; silenzio, vincent; kautz, henry title: the relationship between deteriorating mental health conditions and longitudinal behavioral changes in google and youtube usages among college students in the united states during covid-19: observational study date: 2020-09-05 journal: nan doi: nan sha: doc_id: 120442 cord_uid: qfgoue67 mental health problems among the global population are worsened during the coronavirus disease (covid-19). how individuals engage with online platforms such as google search and youtube undergoes drastic shifts due to pandemic and subsequent lockdowns. such ubiquitous daily behaviors on online platforms have the potential to capture and correlate with clinically alarming deteriorations in mental health profiles in a non-invasive manner. the goal of this study is to examine, among college students, the relationship between deteriorating mental health conditions and changes in user behaviors when engaging with google search and youtube during covid-19. this study recruited a cohort of 49 students from a u.s. college campus during january 2020 (prior to the pandemic) and measured the anxiety and depression levels of each participant. this study followed up with the same cohort during may 2020 (during the pandemic), and the anxiety and depression levels were assessed again. the longitudinal google search and youtube history data were anonymized and collected. from individual-level google search and youtube histories, we developed 5 signals that can quantify shifts in online behaviors during the pandemic. we then assessed the differences between groups with and without deteriorating mental health profiles in terms of these features. significant features included late-night online activities, continuous usages, and time away from the internet, porn consumptions, and keywords associated with negative emotions, social activities, and personal affairs. though further studies are required, our results demonstrated the feasibility of utilizing pervasive online data to establish non-invasive surveillance systems for mental health conditions that bypasses many disadvantages of existing screening methods. globally, mental health problems such as depression, anxiety, and suicide ideations are severely worsened during the coronavirus disease (covid-19) [ [1] [2] [3] , specifically for college students [ 4, [5] [6] [7] ]. yet, current methods for screening mental health issues and identifying vulnerable individuals rely on in-person interviews. such assessments can be expensive, time-consuming, and blocked by social stigmas, not to mention the reluctancy induced by travel restrictions and exposure risks. it has been reported that very few patients in need were correctly identified and received proper mental health treatments on time under the current healthcare system [ 8, 9 ] . even with emerging telehealth technologies and online surveys, the screening requires patients to actively reach out to care providers. at the same time, because of the lockdown enforced by the global pandemic outbreak, people's engagements with online platforms underwent notable changes, particularly in search engine trends [ [10] [11] [12] ], exposures to media reports [ 13, 14 ] , and through quotidian smartphone usages for covid-19 information [ 5 ] . reliance on the internet has significantly increased due to the overnight change in lifestyles, for example, working and remote learning, imposed by the pandemic on society. the sorts of content consumed, the time and duration spent online, and the purpose of online engagements may be influenced by covid-19. furthermore, the digital footprints left by online interactions may reveal information about these changes in user behaviors. most importantly, such ubiquitous online footprints may provide useful signals of deteriorating mental health profiles of users during covid-19. they may capture insights into what was going on in the mind of the user through a non-invasive manner, especially since google and youtube searches are short and succinct and can be quite rich in providing the in the moment cognitive state of a person. on one hand, online engagements can cause fluctuations in mental health. on the other hand, having certain mental health conditions can cause certain types of online behaviors. this opens up possibilities for potential healthcare frameworks that leverage pervasive computing approaches to monitor mental health conditions and deliver interventions on-time. extensive researches have been conducted on a population level, correlating mental health problems with user behaviors on social platforms [ 15, 16 ] , especially among young adolescents. researchers monitored twitter to understand mental health profiles of the general population such as suicide ideations [ 17 ] and depressions [ 18 ] . similar researches have been done with reddit, where anxiety [ 19 ] , suicide ideations [ 17 ] , and other general disorders were studied [ 20, 21 ] . another popular public platform is facebook, and experiments have been done studying anxiety, depression, body shaming, and stress online [ 22, 23 ] . however, such studies were limited to macro observations and failed to identify individuals in need of mental health assistance. in addition, it has been shown that college student communities rely heavily on youtube for both academic and entertainment purposes [ 24, 25 ]. yet, abundant usages may lead to compulsive youtube engagements [ 26 ] , and researchers have found that social anxiety is associated with youtube consumptions in a complex way [ 27 ] . it has been shown that online platforms preserve useful information about the mental health conditions of users, and covid-19 is jeopardizing the mental wellbeing of the global community. thus, we demonstrate the richness of online engagement logs and how it can be leveraged to uncover alarming mental health conditions during covid-19. in this study, we aim to examine whether the changes in user behaviors during covid-19 have a relationship with deteriorating mental health profiles. we focus on google search and youtube usages, and we investigate if the behavior shifts when engaging with these two platforms signify worsened mental health conditions. we hypothesize that late-night activities, compulsive and continuous usages, time away from online platforms, porn and news consumptions, and keywords related to health, social engagements, personal affairs, and negative emotions may play a role in deteriorating mental health conditions. the scope of the study covers undergraduate students in the u.s. we envision this project as a pilot study: it may lay a foundation for mental health surveillance and help delivery frameworks based on pervasive computing and ubiquitous online data. compared to traditional interviews and surveys, such a non-invasive system may be cheaper, efficient, and avoid being blocked by social stigmas while notifying caregivers on-time about individuals at risk. we recruited a cohort of undergraduate students, all of whom were at least 18 years old and have an active google account for at least 2 years, from the university of rochester river campus, rochester, ny, u.s.a. participation was voluntary, and individuals had the option to opt-out of the study at any time, although we did not encounter any such cases. we collected individual-level longitudinal online data (google search and youtube) in the form of private history logs from the participants. for every participant, we measured the depression and anxiety levels via the clinically validated patient health questionnaire-9 (phq-9) and generalized anxiety disorder-7 (gad-7), respectively. basic demographic information was also recorded. there were in total two rounds of data collection: the first round during january 2020 (prior to the pandemic) and the second round during may 2020 (during the pandemic). during each round, for each participant, the anxiety and depression scores were assessed, and the change in mental health conditions was calculated in the end. the entire individual online history data up untill the date of participation was also collected in both rounds from the participants. figure 2 gives an illustration of the recruitment timeline and two rounds of data collections. all individuals participated in both rounds and were compensated with 10-dollar amazon gift cards during each round of participation. given the sensitivity and proprietary nature of private google search and youtube histories, we leveraged the google takeout web interface [ 44 ] to share the data with the research team. prior to any data cleaning and analysis, all sensitive information such as the name, email, phone number, social security number, and credit card information was automatically removed via the data loss prevention (dlp) api [ 45 ] of google cloud. for online data and survey response storage, we utilized a hipaacompliant cloud-based secure storing pipeline. the whole study design, pipelines, and survey measurements involved were similar to our previous setup in [ 43 ] and have been approved by the institutional review board (irb) of the university of rochester. the google takeout platform enables users to share the entire private history logs associated with their google accounts, and as long as the account of the user was logged in, all histories would be recorded regardless of which device the individual was using. each activity in google search and youtube engagement logs were timestamped, signifying when the activity happened to the precision of seconds. besides, for each google search, the history log contained the query text input by the user. it also recorded the url if the user directly input a website address to the search engine. for each youtube video watched by the user, the history log contained the url to the video. if the individual directly searched with keyword(s) on the youtube platform, the history log also recorded the url to the search results. in order to capture the change in online behaviors for the participants, we first introduced a set of features that quantifies certain aspects of how individuals interact with google search and youtube. the set of features was calculated for each participant separately. individual-level behavior changes were then obtained by examining the variations of the feature between january to mid-march of 2020 (prior to the outbreak) and mid-march to may of 2020 (after the outbreak). concretely, we defined 5 features and cut the longitudinal data of each participant into two segments by mid-march, around the time of the covid-19 outbreak in the u.s and campus lockdown. the two segments spanned 2.5 months before and after mid-march, respectively, and data before january 2020 was discarded. the same feature was extracted from both segments of data, and the change was calculated. such change was referred to as the behavior shifts during the pandemic and lockdown. figure 2 gives an illustration of data segmentations and feature development pipelines. we defined late-night activities (lna) as the activities happened between 10:00 p.m. and 5:00 a.m. of the next day, regardless of google search or youtube. for each participant, we counted the numbers of late-night activities before ( ()*+#) ) and after the outbreak ( ()*+#) ), respectively. we then calculated the percentage change of late-night activities and used it as a behavior shift feature: for the rest of the study, any mentioned percentage or relative changes of features were calculated the same way as above. we defined inactivity periods as the periods of time where no google search nor youtube activity was performed. we set a threshold of 7 hours, and we identified all the inactivity periods that were longer than 7 hours for each participant from the online data log. moreover, we looked at how these inactivity periods were distributed across 24 hours. we obtained the mid-point hour mark for each inactivity period: for example, an inactivity period started at 11 p.m. and ended at 7 a.m. has a mid-point of 3 a.m. with normalization, we received a discrete distribution of inactivity period midpoints over the 24-hour bins. it represented how the time away from google search and youtube of an individual was distributed in a 24-hour period. such distribution was calculated on the data segments before ( ()*+#) ) and after ( "*$)# ) the outbreak, respectively. figure 1 showcases two normalized inactivity midpoint distributions before and after the outbreak: after the outbreak, most of the inactive periods of participant 1 shifted to later hours of the dawn, which was most likely to be a delay in bedtime; for participant 2, the morning inactivity moved earlier, and new inactive periods during the afternoon appeared after the outbreak. one possible explanation could be that participant 2 started to take naps after noon, resulting in midpoints around 5 p.m. to estimate the difference before and after the outbreak, we calculated the kldivergence [ 46 ] between the two distributions for each participant: equation 2 . the kl divergence of inactivity distributions before and after the covid-19 outbreak. the kl-divergence is strictly greater than or equals to 0, and it equals to 0 only when the two distributions are identical. we defined a short event interval (sei) as the period of time that is less than 5 minutes between two adjacent events. it usually occurs when one is consuming several youtube videos or searching for related content in a roll. we counted the total numbers of short event intervals for each participant before ( ()*+#) ) and after ( "*$)# ) the outbreak, respectively. we calculated the percentage change of sei the same way as equation 1 and used it as a behavioral feature. the linguistic inquiry and word count (liwc) is a toolkit used to analyze various emotions, cognitive processes, social concerns, and psychological dimensions in a given text by counting the numbers of specific words [ 47 ] . it has been widely applied in researches involving social media and mental health. for the complete list of linguistic and psychological dimensions liwc measures, see [ 47(pp3-4) ]. we segmented the data log for each participant by mid-march as two blobs of texts and analyzed the words using liwc: for google search, we input the raw query text; for youtube, we input the video title. we considered the 'personal concerns', 'negative emotion', 'health/illness', and 'social words' liwc dimensions. liwc categorized words associated with work, leisure, home, money, and religion as 'personal concerns'. in the 'negative emotion' dimension, liwc included words related to anxiety, anger, and sadness. whereas, in the 'social words' dimension, liwc included family, friends, and gender references. the liwc output the count of words falling in each dimension among the whole text. we quantified the shift in behavior by calculating the percentage change of words in each dimension after the outbreak. we labeled each google search query with a category using the google nlp api [ 48 ]. we utilized the official youtube api to retrieve the information of videos watched by the participants, including the title, duration, number of likes and dislikes, and default youtube category tags. for a comprehensive list of google nlp category labels and default youtube category tags, please refer to [ 49,50 ]. there were several categories overlapping with the liwc dimensions, such as 'health' and 'finance', and we regarded the liwc dimensions as a more well-studied standard. instead, we focused on the number of activities belonging to the 'adult' and 'news' categories, which were not presented in the liwc. we calculated the relative changes of activities in these two categories as the behavior shifts for each participant, the same as equation 1. there were in total 5 scalar continuous dependent variables measuring various aspects of the changes in online behavior for each participant, as defined above. these variables were extracted from two segments of the online data logs, namely the data before and after the pandemic outbreak. for the inactivity periods, the measurement was the kl-divergence between inactivity distributions. for the rest 4 behavioral features, the measurements were all in percentage changes. for both rounds of the data collection, anxiety levels were assessed using the gad-7 survey, and depression levels were assessed using the phq-9 survey. with two rounds of surveys reported before and after the outbreak, the change in mental health conditions of each participant was obtained. according to [ 51, 52 ] , an increase greater than or equals to 5 in the gad-7 score may be clinically alarming. therefore, individuals with an increase â³ 5 in gad-7 scores were labeled as the anx group; the rest were labeled as the non-anx group. similarly, as stated in [ 53 ] , an increase greater than or equals to 5 in the phq-9 score may indicate the need for medical interventions. hence, individuals with an increase â³ 5 in phq-9 scores were labeled as the dep group; the rest were labeled as the non-dep group. besides the online data and mental health surveys, we also collected basic demographic information such as school year, gender, and nationality. before any analysis of mental health conditions, in order to eliminate the possibility of annual confounding factors interfering with the shifts in online behaviors, twotailed paired independent t-tests were performed. we inspected that, in terms of the five quantitative features, whether the online behavior changes happened every year, such as due to seasonal factors, or only during covid-19 for the whole study population. as mentioned above, we collected the entire google history log back to the registration date of the google accounts of all participants. thus, we computed the online behaviors changes in both 2020 and 2019 for all participants, spanning 2.5 months before and after the mid-march of each year. the behavior changes were dependent between 2020 and 2019 for the same participant. viewing the cohort as a whole and measured twice, two-tailed paired independent t-tests were performed on all 5 behavior features. for the main experiment, chi-square tests were first performed to investigate the differences in demographics: school year, gender, and nationality. after that, analyses of covariance were conducted to explore the discrepancy between the dep and non-dep groups with each of the 5 online behavior features while controlling significant demographic covariates. the same was performed between the anx and non-anx groups. notice that, in this observational study, the independent variable was the binary group, i.e., whether or not the individual had a significant increase in the gad-7 (or phq-9) score. the dependent variables were the 5 behavior changes extracted from the longitudinal individual online data. experiments were carried out in a one-on-one fashion: anxiety or depression condition was the single independent variable, and one of the 5 online behavior changes was the single dependent variable each time. since multiple hypotheses were tested and some dependent variables might be moderately correlated, a holm's sequential bonferroni procedure was performed with an original significance level a=0.05 to deal with the family-wise error rates. we recruited 49 (n=49) participants in total, and all of them participated in both rounds of the study (response rate=100%). on average, each participant made 2,446 (95% ci 2,120.22-2,481.75) google searches and 2,985 (95% ci 2,576.57-3,393.43) youtube interactions from january to march 14 th , and 2578 (95% ci 2,165.16-2,990.96) google searches and 3146 (95% ci 2,758.79-3,533.24) youtube interactions from march 14 th to the end of may. of the 49 participants, 41% (n=20) of them reported an increase in the phq-9 score â³ 5 (the dep group); 45% (n=22) of them reported an increase in the gad-7 score â³ 5 (the anx group). 37% (n=18) of the participants belonged to the anx and dep group simultaneously. of the 49 participants, 61% (n=30) of the them were female; 35% (n=17) of the them were male; the rest 4% (n=2) reported non-binary genders. first and secondyear students occupied 63% (n=31) of the whole cohort, and the rest were third and fourth-year students (n=24). 80% (n=39) of the participants were u.s. citizens, and the rest (n=10) were international students. a complete breakdown of demographics and group separations are given in table 1 . the two-tailed paired independent t-tests mentioned at the beginning of statistical analysis was designed to rule out seasonal factors in online behavior changes but focus on covid-19 before any of the main experiments, and they reported p<.001 for all 5 quantitative features. hence, the presence of annual or seasonal factors accountable for online behavior changes was neglectable, and it was safe to carry out the following main experiment. this is consistent with one of the main conclusions in [ 5 ] that, when comparing the longitudinal data between different years, behaviors during covid-19 shifted drastically. for each group (anx, non-anx, dep, and non-dep), the average percentage changes in late night activities, short event intervals, liwc attributes, and google search and youtube categories were all positive increases. analyses of covariance were performed to investigate the online behavior differences between the dep and the non-dep groups, ruling out the gender factor. we dummy-coded the categorical gender factor as a continuous covariate. for late night activities, the dep group (mean=9.70%, 95% ci 8.72%-10.68%) had a higher relative increase than the non-dep group (mean=7.54%, 95% ci 6.59%-8.49%), and a significant difference was found (p=. 005 figure 3 shows the distributions of the percentage increases in online behavior features except for the inactivity divergence in the two groups. similar trends were found between the anx and non-anx groups, partially due to the overlapping with the dep and non-dep populations. for late night activities (p=.001, !"#$%"& ' =0.231, f1,46=13.85), the anx group (mean=9.82%, 95% ci 9.04%-10.61%) had a higher percentage increase than the non-anx group (mean=7.28%, 95% ci 6.27%-8.29%). for inactivity periods (p=.01, !"#$%"& ' =0.135, f1,46=7. 19) , the anx group (mean=0.96, 95% ci 0.76-1.16) had a lower divergence, i.e., fewer alterations in the pattern of inactive periods in a 24-hour period, than the non-anx group (mean=1.27, 95% ci 1.13-1.42). the anx group (mean=18.09%, 95% ci 16.22%-19.95%) had more increase in short event intervals than the non-anx group (mean=14.64%, 95% ci13.21%-16.06%), and a significant difference was found (p=.007, !"#$%"& ' =0.149, f1,46=8.05). for the liwc attributes, the anx group (mean=9.15%, 95% ci 7.88%-10.41%) had a higher relative increase in 'personal concern' keywords than the non-anx group (mean=7.37%, 95% ci 6.61%-8.14%), and this difference was statistically significant (p=.02, !"#$%"& ' =0.115, f1,46=5.99). we found a similar result for 'negative words' (p=.01, !"#$%"& ' =0.125, f1,46=6.59) where the anx group (mean=4.12%, 95% ci 3.48%-4.76%) had higher usages than the non-anx group (mean=2.98%, 95% ci 2.53%-3.43%). 'health/illness' (p=. 52 =0.003, f1,46=0.14) content showed any significant group difference. for more details, see table 3 . figure 4 shows the distributions of the percentage increases in online behavior features except for the inactivity divergence in the two groups. in this study, we collected longitudinal individual-level google search and youtube data from college students, and we measured their anxiety (gad-7) and depression (phq-9) levels before and after the outbreak of covid-19. we then developed explainable features from the online data logs and quantified the online behavior shifts of the participants during the pandemic. we also calculated the change in mental health conditions for all participants. our experiment examined the differences between groups with and without deteriorating mental health profiles in terms of these online behavior features. to the best of our knowledge, we are the first to conduct observational studies on how mental health problems and google search and youtube usages of college students are related during covid-19. our results showed significant differences between groups of college students with and without worsened mental health profiles in terms of online behavior changes during the pandemic. the features we developed based on online activities were all explainable and preserved certain levels of interpretability. for example, the short event intervals and inactivity periods measured the consecutive usages and time away from google search and youtube, which were inspired by previous studies on excessive youtube usages [ 26 ] , internet addictions [ 54 ] , and positive associations with social anxiety among college students [ 27 ] . our results indicated that individuals with meaningful increasing anxiety or depressive disorders during the pandemic tended to have long usage sessions (multiple consecutive activities with short time intervals) when engaging with google search and youtube. moreover, anx and dep individuals tended to maintain their regular time-awayfrom-internet patterns regardless of the lockdown as the kl-divergence was low. one possible reason could be that depressed people tend to spend more time at home as regular lifestyles [ 40, 55 ] , and thereby, after the lockdown, the living environment did not alter much. we further found that the majority of the inactivity periods longer than 7 hours had midpoints around 5 to 6 a.m. for all individuals, which were most likely to be the sleeping period. well-established previous researches stated that depressed individuals have more disrupted sleeping patterns and less circadian lifestyles [ 40, 56, 57 ], but they are not validated for special periods such as covid-19. we instead focused on comparing the distributions of time away from google before and after the outbreak of covid-19, and we had an emphasis on the behavior changes of groups with and without worsened mental disorders. besides, the increase in late night activities corresponded with previous studies in sleep deprivation and subsequent positive correlations with mental health deteriorations [ 58, 59 ] . our results demonstrated that individuals with significant worsened anxiety or depressive symptoms during the pandemic were indeed likely to stay up late and engage more online. the above three features captured the temporal aspects of user online behaviors, and they have shown statistically significant differences between groups. additionally, our analysis found that there was a significant difference in the amount of adult and porn consumption between individuals with and without worsening depression, which adheres to previous findings that people suffering from depression and loneliness are likely to consume more pornographies [ 60, 61 . these attributes captured the semantic aspect of user online behaviors. the prevalence of personal affair, social activity, and negative keywords as well as porn consumption have shown statistically significant differences between groups. many researchers have reported that there has been a significant boost in health and news-related topics, at the population level, in various online platforms during covid-19. this is partly due to additional measures taken by individuals, various stakeholders, and agencies with regards to preventive measures [ 11, 35, 36 ], daily statistics [ 10, 12, 13 ], and healthcare (mis)information [ 34, 36, 37 ], however, unlike many, our investigation was carried out considering individual-level google search and youtube engagement logs, and our analysis did not reveal any significant spikes in 'news' and 'health/illness' category between the groups of individuals with deteriorating anxiety and depression during the pandemic. one possible explanation for such observation can be due to the target population (college students) of our study who may prefer to follow news from other popular platforms such as social media. finally, covid-19 has shaken the foundation of human society and forced us to alter daily lifestyles. the world was not ready for such a viral outbreak. since there is no cure for covid-19, it, or an even more deadly viral disease, may resurface at different capacities in the near future. society may be forced to rely on technologies even more and employ remote learning, working, and socializing for a longer period of time. it is important that we learn from our experience of living through the initial covid-19 outbreak and take necessary measures to uncover the changes in online behaviors, investigating how that can be leveraged to understand and monitor various mental health conditions of individuals in the least invasive manner. furthermore, we hope our work paves the path for technology stakeholders to consider incorporating various mental health assessment monitoring systems using user engagements, following users' consent in a privacy-preserving manner. they can periodically share the mental health monitoring assessment report with respective users based on their online activities, education, and informing users about their current mental health. this can eventually encourage individuals to acknowledge the importance of mental health and take better care of themselves. first, while most of the online behavioral features we developed showed significant differences between groups of students with and without deteriorating anxiety and depressive disorders during covid-19, our study cohort only represented a small portion of the whole population suffering from mental health difficulties. therefore, further studies are required to investigate if the significant behavioral changes still hold among more general communities, not limiting to college students. nonetheless, we argue that the explainable features we constructed, such as late-night activities, continuous usages, inactivity, pornography, and certain keywords, can remain behaviorally representative and be applied universally across experiments exploring the relationship between mental health and online activities during the pandemic. second, in this work, we studied the relationship between user online behaviors and the fluctuations in mental health conditions during covid-19. any causal relationship between online behavior and mental disorders is beyond the scope of this work. as one can readily imagine, online behavioral changes could both contribute to or be caused by deteriorating anxiety or depressive disorders. moreover, though we included preliminary demographic information as covariates, there remains the possibility of other confounding factors. in fact, both the shifts in online behaviors and deteriorating mental health profiles may be due to common factors such as living conditions, financial difficulties, and other health problems during the pandemic. nor there was any causal direction implied between covid-19 and online behavior changes, which was introduced in the first paragraph of statistical analysis as a precaution before the main experiments. albeit a pilot study, our results indicated that it is possible to build an anxiety and depression surveillance system based on passively collected private google data histories during covid-19. such non-invasive systems shall be subject to rigorous data security and anonymity checks. necessary measures need to be in place to ensure personal safety and privacy concerns when collecting sensitive and proprietary data such as google search logs and youtube histories. even in pilot studies, participants shall preserve full rights over their data: they may choose to opt-out of the study at any stage and remove any data shared in the system. moreover, anonymity and systematic bias elimination shall be enforced. as an automatic medical screening system based on pervasive data, it has been extensively studied that such frameworks are prone to implicit machine learning bias during data collection or training phases [ [66] [67] [68] . black-box methods should be avoided as they are known to be vulnerable to adversarial attacks and produce unexplainable distributional representations [ 69, 70 ] . anonymizing data and obscuring identity information should be the first step in data debiasing. in the end, to what extent should caregivers trust a clinical decision made by machines remains an open question. we believe that possible pervasive computing frameworks shall play the role of a smart assistant, at most, to the care providers. any final intervention or help delivery decision should be made by healthcare professionals who understand both the mental health problems and the limitations of automatic detection systems in clinical settings. the outbreak of covid-19 coronavirus and its impact on global mental health mental health and the covid-19 pandemic covid-19 and mental health: a review of the existing literature students under lockdown: comparisons of students' social networks and mental health before and during the covid-19 crisis in switzerland mental health and behavior of college students during the early phases of the covid-19 pandemic: longitudinal smartphone and ecological momentary assessment study psychological state of college students during covid-19 social science research network the psychological impact of the covid-19 epidemic on college students in china prevalence, severity, and comorbidity of twelve-month dsm-iv disorders in the national comorbidity survey replication (ncs-r) failure and delay in initial treatment contact after first onset of mental disorders in the national comorbidity survey replication tracking covid-19 in europe: infodemiology approach. jmir public health and surveillance use of google trends to investigate loss-ofsmell-related searches during the covid-19 outbreak binge watching behavior during covid 19 pandemic: a cross-sectional, cross-national online survey mental health problems and social media exposure during covid-19 outbreak social networking sites, depression, and anxiety: a systematic review association between social media use and depression among u.s. young adults. depression and anxiety discovering shifts to suicidal ideation from mental health content in social media social media as a measurement tool of depression in populations detecting anxiety through reddit online social networks in health care: a study of mental disorders on reddit let me tell you about your mental health!": contextualized classification of reddit posts to dsm-5 for web-based intervention the stress-buffering effect of self-disclosure on facebook: an examination of stressful life events, social support, and mental health among college students youtube in the classroom: helpful tips and student perceptions social media as a complementary learning tool for teaching and learning: the case of youtube compulsive youtube usage: a comparison of use motivation and personality effects the relations between youtube addiction, social anxiety and parasocial relationships with youtubers: a moderated-mediation model based on a cognitive-behavioral framework suicide risk and prevention during the covid-19 pandemic. the lancet psychiatry using internet-based psychological measurement to capture the deteriorating community mental health profile during covid-19: observational study multidisciplinary research priorities for the covid-19 pandemic: a call for action for mental health science. the lancet psychiatry managing mental health challenges faced by healthcare workers during covid-19 pandemic mental health care for medical staff in china during the covid-19 outbreak. the lancet psychiatry patients with mental health disorders in the covid-19 epidemic. the lancet psychiatry youtube as a source of information on covid-19: a pandemic of misinformation? the role of youtube and the entertainment industry in saving lives by educating and mobilizing the public to adopt behaviors for community mitigation of covid-19: successive sampling design study preventive behaviors conveyed on youtube to mitigate transmission of covid-19: cross-sectional study. jmir public health and surveillance misinformation on social media: experimental evidence for a scalable accuracy-nudge intervention sense and sensibility: characterizing social media users regarding the use of controversial terms for covid-19 the relationship between mobile phone location sensor data and depressive symptom severity mobile phone sensor correlates of depressive symptom severity in daily-life behavior: an exploratory study objective assessment of depressive symptoms with machine learning and wearable sensors data tracking depression dynamics in college students using mobile phone and wearable sensing detecting low self-esteem in youths from web search data on information and sufficiency the development and psychometric properties of liwc2015 a brief measure for assessing generalized anxiety disorder: the gad-7 psychometric properties of the generalized anxiety disorder scale-7 (gad-7) in outpatients with anxiety and mood disorders enhancing the clinical utility of depression screening internet addiction in students: prevalence and risk factors association between lifestyle activity and depressed mood among home-dwelling older people: a community-based study in japan circadian rhythms and depression: human psychopathology and animal models circadian pattern of motor activity in major depressed patients undergoing antidepressant therapy: relationship between actigraphic measures and clinical course sleep in adolescence: physiology, cognition and mental health reciprocal relationships between daily sleep and mood: a systematic review of naturalistic prospective studies understanding associations between personal definitions of pornography, using pornography, and depression youtube as a source of information on immunization: a content analysis nonverbal social withdrawal in depression: evidence from manual and automatic analyses. image and vision computing childhood social withdrawal, interpersonal impairment, and young adult depression: a mediational model artificial intelligence, bias and clinical safety potential biases in machine learning algorithms using electronic health record data a systematic review of machine learning models for predicting outcomes of stroke with structured data exploiting the vulnerability of deep learning-based artificial intelligence models in medical imaging: adversarial attacks understanding adversarial attacks on deep learning based medical image analysis systems. pattern recognition this research was supported in part by grant w911nf-15-1-0542 and w911nf-19-1-0029 with the us defense advanced research projects agency (darpa) and the army research office (aro). we acknowledge the contributions by michael giardino, adira blumenthal, and ariel hirschhorn at the beginning phase of the project. key: cord-305195-e41yfo89 authors: rainwater-lovett, kaitlin; rodriguez-barraquer, isabel; moss, william j. title: viral epidemiology: tracking viruses with smartphones and social media date: 2016-02-12 journal: viral pathogenesis doi: 10.1016/b978-0-12-800964-2.00018-5 sha: doc_id: 305195 cord_uid: e41yfo89 the science of epidemiology has been developed over the last 200 years, using traditional methods to describe the distribution of diseases by person, place, and time. however, in the last several decades, a new set of technologies has become available, based on the methods of computer sciences, systems biology, and the extraordinary powers of the internet. technological and analytical advances can enhance traditional epidemiological methods to study the emergence, epidemiology, and transmission dynamics of viruses and associated diseases. social media are increasingly used to detect the emergence and geographic spread of viral disease outbreaks. large-scale population movement can be estimated using satellite imagery and mobile phone use, and fine-scale population movement can be tracked using global positioning system loggers, allowing estimation of transmission pathways and contact patterns at different spatial scales. advances in genomic sequencing and bioinformatics permit more accurate determination of viral evolution and the construction of transmission networks, also at different spatial and temporal scales. phylodynamics links evolutionary and epidemiological processes to better understand viral transmission patterns. more complex and realistic mathematical models of virus transmission within human and animal populations, including detailed agent-based models, are increasingly used to predict transmission patterns and the impact of control interventions such as vaccination and quarantine. in this chapter, we will briefly review traditional epidemiological methods and then describe the new technologies with some examples of their application. the science of epidemiology has been developed over the last 200 years, using traditional methods to describe the distribution of diseases by person, place, and time. however, in the last several decades, a new set of technologies has become available, based on the methods of computer sciences, systems biology, and the extraordinary powers of the internet. technological and analytical advances can enhance traditional epidemiological methods to study the emergence, epidemiology, and transmission dynamics of viruses and associated diseases. social media are increasingly used to detect the emergence and geographic spread of viral disease outbreaks. large-scale population movement can be estimated using satellite imagery and mobile phone use, and fine-scale population movement can be tracked using global positioning system (gps) loggers, allowing estimation of transmission pathways and contact patterns at different spatial scales. advances in genomic sequencing and bioinformatics permit more accurate determination of viral evolution and the construction of transmission networks, also at different spatial and temporal scales. phylodynamics links evolutionary and epidemiological processes to better understand viral transmission patterns. more complex and realistic mathematical models of virus transmission within human and animal populations, including detailed agent-based models, are increasingly used to predict transmission patterns and the impact of control interventions such as vaccination and quarantine. in this chapter, we will briefly review traditional epidemiological methods and then describe the new technologies with some examples of their application. insight into the epidemiology of viral infections long preceded the recognition and characterization of viruses as communicable agents of disease in humans and animals, extending at least as far back as the treatise of abu becr (rhazes) on measles and smallpox in the tenth century. successful efforts to alter the epidemiology of viral infections can be traced to the practice of variolation, the deliberate inoculation of infectious material from persons with smallpox (see chapter on the history of viral pathogenesis). documented use of variolation dates to the fifteenth century in china. edward jenner greatly improved the practice of variolation in 1796 using the less-virulent cowpox virus, establishing the field of vaccinology. an early example of rigorous epidemiological study prior to the discovery of viruses was the work of the danish physician peter panum who investigated an outbreak of measles on the faroe islands in 1846. through careful documentation of clinical cases and contact histories, panum provided evidence of the contagious nature of measles, accurate measurement of the incubation period, and demonstration of the long-term protective immunity conferred by measles. the discovery of viruses as "filterable agents" in the late-nineteenth and early twentieth centuries greatly enhanced the study of viral epidemiology, allowing the characterization of infected individuals, risk factors for infection and disease, and transmission pathways. traditional epidemiological methods measure the distribution of viral infections, diseases, and associated risk factors in populations in terms of person, place, and time using standard measures of disease frequency, study designs, and approaches to causal inference. populations are often defined in terms of target and study populations, and individuals within study populations in terms of exposure and outcome status. the purpose of much traditional epidemiological research is to quantify the strength of association between exposures and outcomes by comparing characteristics of groups of individuals. exposures or risk factors include demographic, social, genetic, and environmental factors, and outcomes include infection or disease. in viral epidemiology, infection status is determined using diagnostic methods to detect viral proteins or nucleic acids, and serologic assays to measure immunologic markers of exposure to viral antigens. infection status can be defined as acute, chronic, or latent. standard measures of disease frequency include incidence, the number of new cases per period of observation (e.g., 1000 person-years), and prevalence, the number of all cases in a defined population and time period. prevalence is a function of both incidence and duration of infection and can increase despite declining incidence, as observed with the introduction of antiretroviral therapy for human immunodeficiency virus (hiv) infection in the united states. although the number of new cases of hiv infection declined, the prevalence of hiv infection increased as treated individuals survived longer. commonly used study designs include, l cross-sectional studies in which individuals are sampled or surveyed for exposure and disease status within a narrow time frame, l cohort studies in which exposed and unexposed individuals are observed over time for the onset of specified outcomes, l case-control studies in which those with and without the outcome (infection or disease) are compared on exposure status, and l clinical trials in which individuals are randomized to an exposure such as a vaccine or drug and observed for the onset of specified outcomes appropriate study design, rigorous adherence to study protocols, and statistical methods are used to address threats to causal inference (i.e., whether observed associations between exposure and outcome are causal), such as bias and confounding. much can be learned about the epidemiology of viral infections using such traditional methods and many examples could be cited to establish the importance of these approaches, including demonstration of the mode of transmission of viruses by mosquitoes (e.g., yellow fever and west nile viruses), the causal relationship between maternal viral infection and fetal abnormalities (e.g., rubella virus and cytomegalovirus), and the role of viruses in the etiology of cancer (e.g., epstein-barr and human papilloma viruses). the epidemiology of communicable infectious diseases is distinguishable from the epidemiology of noncommunicable diseases in that the former must account for "dependent happenings." this term was introduced by ronald ross to capture the fact that infectious agents are transmitted between individuals or from a common source. traditional epidemiological and statistical methods often assume disease events in a population are independent of one another. in infectious disease epidemiology, individuals are defined in terms of susceptible, exposed, infectious, and recovered or immune. key characteristics of viral infections that determine the frequency and timing of transmission, and thus the epidemiology, include the mode of transmission (e.g., respiratory, gastrointestinal, sexual, bloodborne, and vector-borne), whether infection is transient or persistent, and whether immunity is short or long lasting. temporal changes in the transmission dynamics of viral infection can be displayed with epidemic curves, by plotting the number or incidence of new infections over time to demonstrate outbreaks, seasonality, and the response to interventions. key metrics in infectious disease epidemiology that capture the dependent nature of communicable diseases include: (1) the latent period, the average time from infection to the onset of infectiousness; (2) the infectious period, the average duration of infectiousness; (3) the generation time, the average period between infection in one individual and transmission to another; and (4) the basic reproductive number (r 0 ), the average number of new infections initiated by a single infectious individual in a completely susceptible population over the course of that individual's infectious period. if r 0 is larger than one, the number of infected individuals and hence the size of the outbreak will increase. if r 0 is smaller than one, each infectious individual infects on average less than one other individual and the number of infected individuals will decrease and the outbreak ceases. the reproductive number (r) is a function not only of characteristics of the viral pathogen (e.g., mode of transmission), but also the social contact network within which it is transmitted and changes over time in response to a decreasing number of susceptible individuals and control interventions. an important concept related to the interdependence of transmission events is herd immunity, the protection of susceptible individuals against infection in populations with a high proportion of immune individuals because of the low probability of an infectious individual coming in contact with a susceptible individual. the concepts and methods of infectious disease epidemiology provide the tools to understand changes in temporal and spatial patterns of viral infections and the impact of interventions. traditional epidemiological methods provide powerful analytical approaches to measure associations between exposures (risk factors) and outcomes (infection or disease). recent technological advances enhance these methods and permit novel approaches to investigate the emergence, epidemiology, and transmission dynamics of viruses and associated diseases. expanded access to the internet and social media has revolutionized outbreak detection and viral disease surveillance by providing novel sources of data in real time (chunara, 2012) . traditional epidemiologic surveillance systems rely on standardized case definitions, with individual cases typically classified as suspected, probable, or confirmed based on the level of evidence. confirmed cases require laboratory evidence of viral infection. surveillance systems are either active or passive. active surveillance involves the purposeful search for cases within populations whereas passive surveillance relies on routine reporting of cases, typically by health care workers, health care facilities, and laboratories. data acquired through active surveillance are often of higher quality because of better adherence to standardized case definitions and completeness of case ascertainment but are more expensive and resource intensive. however, both active and passive surveillance are prone to delays in data reporting. the major advantage of using the internet and social media to monitor disease activity is that the signal can be detected without the lag associated with traditional surveillance systems. influenza is the most common viral infection for which the internet and social media have been used for disease surveillance because of its high incidence, wide geographic distribution, discrete seasonality, short symptomatic period, and relatively specific set of signs and symptoms. however, the internet and social media have several limitations compared to traditional active and passive surveillance systems and complement rather than replace these methods. these limitations include lack of specificity in the "diagnosis," and waxing and waning interest and attention in social media independent of disease frequency. in 2008, the internet company google developed a webbased tool called google flu trends, for early detection of influenza outbreaks. google flu trends is based on the fact that millions of people use the google search engine each day to obtain health-related information (ginsberg, 2009 ). logs of user key words for pathogens, diseases, symptoms, and treatments, as well as information on user location contained in computer internet protocol (ip) addresses, allow temporal and spatial analyses of trends in search terms ( figure 1 ). early results suggested that google flu trends detected regional outbreaks of influenza 7-10 days before conventional surveillance by the centers for disease control and prevention (carneiro, 2009 ). however, accurate prediction was not as reliable as initially thought, and google estimates did not closely match measured activity during the 2012-2013 influenza season. google now reevaluates estimates using data from traditional surveillance systems (specifically those of the centers for disease control and prevention) to refine model and parameter estimates. these refinements more accurately capture the start of the influenza season, the time of peak influenza virus transmission, and the severity of the influenza season. a similar approach, called google dengue trends, is used to track dengue virus infections by aggregating historical logs of anonymous online google search queries associated with dengue, using the methods developed for google flu trends. early observations suggest google queries are correlated with national-level dengue surveillance data, and this novel data source may have the potential to provide information faster than traditional surveillance systems ( figure 2 ). other internet sources are being explored to enhance viral surveillance. wikipedia is a free, online encyclopedia written collaboratively by users and is one of the most commonly used internet resources since it was started in 2001. as with google searches, the use of disease-specific queries to wikipedia are expected to correlate with disease activity. the number of times specific influenza-related wikipedia sites were accessed provided accurate estimates of influenza-like illnesses in the united states 2 weeks earlier than standard surveillance systems and performed better than google flu trends (mciver, 2014) . similarly, social media data are being evaluated for surveillance purposes. twitter is a free social networking service that enables users to exchange text-based messages of up to 140 characters known as tweets. as with google flu trends, the number of tweets related to influenza activity is correlated with the number of symptomatic individuals. several published studies reported correlations between twitter activity and reported influenza-like illnesses (chew, 2010; signorini, 2011 ; figure 3 ). limitations to using social media, such as twitter, to monitor disease activity are illustrated by the ebola virus outbreak in west africa in early 2014. despite the fact that ebola had not yet occurred in the united states, posts to twitter on ebola rose dramatically, likely in response to intense media coverage and fear. clearly, such tweets could not be interpreted to indicate ebola disease activity in the united states. studies reporting misleading associations, or the lack of correlation between social media and disease activity, are rarely published, providing a cautionary note. while initial efforts using data from the internet for viral disease surveillance offer promising results, concerns have been raised regarding the utility and robustness of these approaches (lazer, 2014) . integration into existing surveillance frameworks will be necessary to maximize the utility of these data streams. the internet allows rapid processing and communication of health-related information, including the aggregation and display of surveillance data for viral infections. traditional surveillance networks can be linked through the internet to allow rapid integration and dissemination of information. information on viral disease outbreaks available through internet postings of health care agencies such as the world health organization (who) and centers for disease control and prevention (cdc), as well as press reports and blogs, can provide data that are more current than traditional surveillance systems. information from these online sources can be made available to a large, global audience. several of the most commonly used surveillance sites report animal as well as human diseases (see sidebar 1 and figure 4 ). mapping spatial patterns of disease and relationships with environmental variables preceded the development of modern epidemiology. the classic example is john snow's hand-drawn map of london cholera cases of 1854. however, routine mapping of health data only became commonplace in the 1990s after desktop geographic information systems became widely available. combined with satellite imagery and remotely sensed environmental and ecological data, spatial mapping of viral infections is a powerful tool for surveillance and epidemiological research. spatial epidemiology is typically used to identify and monitor areas of differential risk. an early example was a large outbreak of st. louis encephalitis virus infection in houston, texas in 1964. spatial analysis showed that the outbreak was concentrated in the city center, with lower incidence at the outskirts. further investigation revealed that the city center was associated with the lowest economic strata, unscreened windows, lack of air-conditioning and pools of standing water, factors facilitating virus transmission. investigation into the spatiotemporal dynamics of viral diseases at smaller spatial scales has become promed, the program for monitoring emerging diseases, is an internet-based reporting system established in 1994 that compiles information on outbreaks of infectious diseases affecting humans, animals, and food plants. promed relies on official announcements, media reports, and local observers, including the network of subscribers. a team of experts screen, review, and investigate reports before posting and often provide commentary. reports are distributed by email to direct subscribers and posted on the promed-mail web site. promed-mail currently reaches over 60,000 subscribers in at least 185 countries. started in 2006 by epidemiologists and software developers at boston children's hospital, healthmap monitors disease outbreaks and provides real-time surveillance of emerging public health threats, including viral infections (figure 4) . healthmap organizes and displays data on disease outbreaks and surveillance using an automated process. data sources include online news aggregators, eyewitness reports, expertcurated discussions and validated official reports. google flu trends http://www.google.org/flutrends google dengue trends http://www.google.org/denguetrends healthmap http://healthmap.org promed http://www.promedmail.org possible with increasing availability of global positioning systems devices and geocoding algorithms. such studies have revealed spatial heterogeneity in the local transmission of some directly (e.g., hiv and influenza) and indirectly (e.g., dengue and chikungunya) transmitted viruses. for example, clustering analyses of the residential locations of people with dengue in bangkok over a 5-year period showed evidence of localized transmission at distances less than 1 km (salje, 2012; figure 5 ). analyses of data from a large population-based cohort of hiv-infected persons in rakai district, uganda revealed strong within-household clustering of prevalent and incident hiv cases as well clustering of prevalent cases up to 500 m (grabowski, 2014) . beyond descriptive applications, mapping spatiotemporal patterns of viral infections can provide fundamental insights into transmission dynamics at different spatial scales. traveling waves from large cities to small towns were shown to drive the spatiotemporal dynamics of measles in england and wales (xia, 2004) . the incidence of dengue hemorrhagic fever across thailand manifested as a traveling wave emanating from bangkok and moving radially at a speed of 148 km/month (cummings, 2004) . insight into the spatial epidemiology of viral infections and associations with environmental risk factors can be greatly enhanced when information on the spatial location of cases is combined with remotely sensed environmental data (rodgers, 2003) . the spatial coordinates of cases can be overlaid on satellite imagery to demonstrate relationships with environmental features-such as bodies of water-and formally analyzed using spatial statistical techniques. satellite sensors that detect reflected visible or infrared radiation provide additional information on temperature, rainfall, humidity, and vegetation among other variables, which are particularly important for the transmission dynamics of vector-borne viral infections. satellite data for epidemiologic analyses are provided by a number of sources such as: (1) earth-observing satellites with high spatial resolution (1-4 m) but low repeat frequencies such as ikonos and landsat satellites; (2) oceanographic and atmospheric satellites such as modis and aster with lower spatial resolution (0.25-1 km) that provide images of the earth surface twice a day; and (3) geostationary weather satellites such as geos with large spatial resolution (1-8 km). the statistical relationships between cases and environmental risk factors can be used to construct risk maps. risk maps display the similarity of environmental features in unsampled locations to environmental features in locations where the disease is measured to be present or absent. spatial analysis of the initial cases of west nile virus infection in new york city in 1999 identified a significant spatial cluster (brownstein, 2002) . using models incorporating measures of vegetation cover from satellite imagery, the risk of west nile virus could be estimated throughout the city. a more recent risk map for west nile virus in suffolk county, new york, was generated with data on vector habitat, landscape, virus activity, and socioeconomic variables derived from publicly available data sets (rochlin, 2011; figure 6 ). population movement plays a crucial role in the spread of viral infections. in the past, quantifying the contribution of movement to viral transmission dynamics at different spatial scales was challenging, due to limited data. as an early example, the impact of restrictions of animal movement on transmission of foot-and-mouth disease in 2001 was estimated, using detailed contact-tracing data from farms in the united kingdom (shirley, 2005) . however, such detailed data are rarely available for patterns of human movement. studies have attempted to model the impact of long-range human movement on the spread of viral diseases using measures such as distance between cities, commuting rates, and data on air travel. this approach has been used to explain regional and interregional spread of influenza viruses. data on air traffic volume, distance between areas, and population sizes have been invoked to describe and predict local and regional spread of chikungunya virus in the americas (tatem, 2012) . new technologies have greatly enhanced the capacity to study the impact of human movement on transmission dynamics of infectious diseases. data from mobile phones and gps loggers can be used to characterize individual movement patterns and the time spent in different locations (figure 7) . individual movement patterns can be overlaid on risk maps to quantify movement to and from areas of high (sources) and low risk (sinks) as well as to estimate potential contact patterns. gps data loggers generated 2.3 million gps data points to track the fine-scale mobility patterns of 582 residents from two neighborhoods in iquitos, peru, to better understand the epidemiology of viral infections (vazquez-prokopec, 2013) . most movement occurred within 1 km of an individual's home. however, potential contacts between individuals were irregular and temporally unstructured, with fewer than half of the tracked participants having a regular, predictable routine. the investigators explored the potential impact of these temporally unstructured daily routines and contact patterns on the simulated spread of influenza virus. the projected outbreak size was 20% larger as a consequence of these unstructured contact patterns, in comparison to scenarios modeling temporally structured contacts. in addition to identifying individual and environmental characteristics associated with temporal and spatial patterns of viral infections, transmission networks are critical drivers of the dynamics of viral infections. analysis of transmission networks defines the host contact structure within which directly transmitted viral infections spread. network theory and analysis are complex subjects with a long history in mathematics and sociology, but have recently been adapted by infectious disease epidemiologists. the epidemiologic study of social networks is facilitated by unique study designs, including snowball sampling or respondent-driven sampling, in which study participants are asked to recruit additional participants among their social contacts. differing sexual contact patterns serve as an example of the importance of contact networks to the understanding of viral epidemiology. concurrent sexual partnerships amplify the spread of hiv compared with serial monogamy. this could partially explain the dramatic differences in the prevalence of hiv in different countries. social networks were shown to affect transmission of the 2009 h1n1 influenza virus, and were responsible for cyclical patterns of transmission between schools, communities, and households. technological advances in quantifying contact patterns, with wearable sensors and the use of viral genetic signatures, have greatly enhanced the ability to understand complex transmission networks. self-reported contact histories and contact tracing are the traditional epidemiological methods to define transmission networks. contact tracing has a long history in public health, particularly in the control of sexually transmitted diseases and tuberculosis, and is critical to the control of outbreaks of viral infections such as the middle east respiratory syndrome coronavirus (mers-cov) and ebola virus. to better understand the nature of human contact patterns, sensor nodes or motes have been used to characterize the frequency and duration of contacts between individuals in settings such as schools and health-care facilities. these technologies offer opportunities to validate and complement data collected using questionnaires and contact diaries. as an example, investigators used wireless sensor network technology to obtain data on social contacts within 3 m for 788 high school students in the united states, enabling construction of the social network within which a respiratory pathogen could be transmitted (salathe, 2010) . the data revealed a high-density network with typical small-world properties, in which a small number of steps link any two individuals. computer simulations of the spread of an influenza-like virus on the weighted contact graph were in good agreement with absentee data collected during the influenza season. analysis of targeted immunization strategies suggested that contact network data can be employed to design targeted vaccination strategies that are significantly more effective than random vaccination. advances in nucleic acid sequencing and bioinformatics have led to major advances in viral epidemiology. population (sanger) sequencing has been the standard method for dna sequencing but is increasingly replaced by deep sequencing in which variants within a viral swarm are distinguished. sequencing allows for the detection of single nucleotide polymorphisms (snps) and nucleotide insertions or deletions ("indels"), analysis of synonymous and nonsynonymous mutations, and phylogenetic analysis (see chapter on virus evolution). sequencing techniques can be applied to both viral and host genomes. snps may be associated with changes in viral pathogenesis, virulence, or drug resistance. molecular techniques applied to pathogens also have been fundamental to the study of the animal origins of many viral infections including hiv and mers. phylogeographic approaches were used to trace the origins of the hiv pandemic to spillover events in central africa (sharp, 2010) . more recently, sequence data were used to track the animal reservoirs of mers-cov associated with the 2014 outbreaks (haagmans, 2014) , and to compare the ebola virus strain circulating in the 2014 west africa outbreak to strains from prior outbreaks (gire, 2014) . epidemiologic studies that probe host genomes can be either candidate gene studies or genome-wide association studies. the goal of these studies is to link specific changes with an increased risk of infection or disease. as an example, a small subset of individuals who failed to acquire hiv infection despite exposure, prompted studies to determine how these individuals differed from those who acquired infection. a 32-base-pair deletion in the human ccr5 gene, now referred to as ccr5-delta 32, accounted for the resistance of these subjects. individuals who are ccr5-delta 32 homozygotes are protected against hiv infection by ccr5tropic hiv strains, while heterozygotes have decreased disease severity. infectious disease epidemiologists are increasingly linking evolutionary, immunologic, and epidemiological processes, a field referred to as phylodynamics voltz, 2013) . because of the high mutation rates of viral pathogens, particularly rna viruses, evolutionary and epidemiological processes take place on a similar timescale (see chapter on virus evolution). according to this framework, phylodynamic processes that determine the degree of viral diversity are a function of host immune selective pressures and epidemiological patterns of transmission ( figure 8 ). intrahost phylodynamic processes begin with molecular characteristics of the virus as well as the host's permissiveness and response to infection. for example, a single amino acid substitution in epstein-barr virus was shown to disrupt antigen presentation by specific human leukocyte antigen polymorphisms (liu, 2014) . this resulted in decreased t-cell receptor recognition and successful viral immune escape. the virus must also induce an "optimal" host immune response to maximize transmission to new hosts. if the virus induces a strong, proinflammatory immune response not balanced by the appropriate anti-inflammatory responses, the host may succumb to the overabundance of inflammation and cannot propagate viral transmission. alternatively, a virus that fails to stimulate an immune response may also replicate uninhibited, overwhelming, and killing the host prior to transmission. selective pressures maximize replication while sustaining transmission between hosts. interhost dynamics are affected by several factors including evolutionary pressures, timescales of infection, viral latent periods, and host population structures. typically, only a small number of virions are transmitted between hosts, creating a genetic bottleneck that limits viral diversity. a virus that mutates to cause highly pathogenic disease but is not transmitted cannot propagate its pathogenicity. cross-immunity between viral strains also precludes the replication of particular viral lineages. influenza vaccine strains require annual changes due to new circulating influenza strains that have escaped immune pressures through high mutation rates and gene re-assortment. the strong selection pressure of cross-immunity is reflected in the short branch lengths in a phylogenetic tree of influenza viruses isolated from infected individuals. thus, the selection of influenza strains for future vaccines is partly determined by cross-immunity to prior circulating strains, because influenza viral strains that circulated in the past may elicit immune protection against currently circulating strains. at the population level, phylodynamic methods have been used to estimate r 0 for hiv and hepatitis c virus, for which reporting and surveillance data are often incomplete (volz, 2013) . phylodynamic and phylogeographic models also have been useful in reconstructing the spatial spread of viruses to reveal hidden patterns of transmission. for example, epidemiological and molecular studies of influenza virus transmission were compared at different spatial scales to highlight the similarities and differences between these data sources (viboud, 2013) . the findings were broadly consistent with large-scale studies of interregional or inter-hemispheric spread in temperate regions with multiple viral introductions resulting in epidemics followed by interepidemic periods driven by seasonal bottlenecks. however, at smaller spatial scalessuch as a country or community-epidemiological studies revealed spatially structured diffusion patterns that were not identified in molecular studies. phylogenetic analyses of gag and env genes were used to assess the spatial dynamics of hiv transmission in rural rakai district, uganda, using data from a cohort of 14,594 individuals residing in 46 communities (grabowski, 2014) . of the 95 phylogenetic clusters identified, almost half comprised two individuals sharing a household. among phylodynamics links evolutionary, immunologic, and epidemiologic processes to explain viral diversity, as shown here for equine influenza virus. for viral evolution, these processes take place on a similar timescale. within host mutations (1) result from an interplay between optimization of viral shedding, immunologic selective pressures and host pathogenicity. transmission bottlenecks and host heterogeneity (2) further determine the population genetic structure of the virus, which in turn influences and is determined by the epidemic dynamics. larger scale spatial dynamics at local, regional, and global levels (3) the remaining clusters, almost three-quarters involved individuals living in different communities, suggesting transmission chains frequently extend beyond local communities in rural uganda. the timescale of infection is also important for viral diversity and transmission dynamics. some viruses are capable of initiating an acute infection that is cleared within days, while other viral infections are chronic and persist for a lifetime. the duration of infection impacts how quickly a virus must be transmitted and has implications for the infectious period and the potential to be transmitted to new hosts. viruses with long latent periods create interhost phylogenetic trees with longer branch lengths. the long duration between infection and transmission permits accumulation of viral changes through many rounds of viral replication before transmission to the next host. examples include hepatitis b virus, hepatitis c virus, and human immunodeficiency virus. availability of computational resources allows widespread use and development of classic approaches to the mathematical modeling of viral transmission dynamics, such as compartmental, metapopulation and network models, to address epidemiologic questions (see chapter on mathematical methods). these models have been used extensively in the study of viral dynamics and to explore the potential impact of control interventions. new sources of high-resolution spatial, temporal, and genetic data create opportunities for models that integrate these data with traditional epidemiological data. such analyses improve estimates of key transmission parameters and understanding of the mechanisms driving virus spread. agent-based models (also known as individual-based models) can now be run using desktop computers, and offer advantages over more traditional mathematical models. because each unit in a population is modeled explicitly in space and time and assigned specific attributes, agent-based models can reproduce the heterogeneity and complexity observed in the real world. more traditional compartmental, differential equation models often require simplifying assumptions that limit applicability. agent-based models have been used to study the spread of viruses in populations as well as the evolution of viruses within and across populations. while agent-based models are intuitive and easy to formulate, these models are often difficult to construct due to the large number of parameters necessary to describe the behavior and interaction between individual units. commercial frameworks that offer large computational power and intuitive user interphases have also become increasingly available. the global epidemic and mobility model (gleam) on the gleamviz platform (www.gleamviz.com), for example, contains extensive data on populations and human mobility, and allows stochastic simulation of the global spread of infectious diseases using user-defined transmission models. our understanding of the epidemiology of viral infections is being revolutionized by the integration of traditional epidemiological information with novel sources of data. l data streams from the internet are promising sources to enhance traditional surveillance but have yet to be fully validated. l molecular data on viral genomic sequences provide unprecedented opportunities to characterize viral transmission pathways. l phylodynamic and phylogeographic models have been used to estimate r 0 , and characterize the spatial spread of viruses. l network analysis reveals hidden patterns of transmission between population subgroups that are not easy to capture with traditional epidemiological methods. l novel analytical and computational resources are playing a key role in integrating information from multiple large data banks. these more comprehensive methods improve our ability to estimate the impact of infection control measures. the combination of traditional and evolving methodologies is closing the gap between epidemiological studies and viral pathogenesis. these developments have laid the foundation for exciting future research that will complement other approaches to the pathogenesis of viral diseases. with these evolving technologies in mind, it is timely to ask: is the world able to control viral diseases more effectively? it is a mixed score card. on the one hand, smallpox has been eradicated and we are on the verge of elimination of wild polioviruses. furthermore, deaths of children under the age of 5 years (which are mainly due to viral and other infectious diseases) have decreased by almost 50% in the last few decades. on the other hand, the aids pandemic continues to rage in low-income countries, with only a slight reduction in the annual incidence of new infections. the united states has not done any better in reducing hiv incidence which has been unchanged for at least 20 years. the 2014-15 ebola pandemic in west africa reflects the limited capacity for dealing with new and emerging viral diseases on a global basis. in conclusion, epidemiological science continues to advance with evolving new technologies, but their application to public health remains a future challenge and opportunity. new technologies for reporting real-time emergent infections google trends: a web-based tool for real-time surveillance of disease outbreaks unifying the epidemiological and evolutionary dynamics of pathogens studying the global distribution of infectious diseases using gis and rs spatial analysis of west nile virus: rapid risk assessment of an introduced vector-borne zoonosis pandemics in the age of twitter: content analysis of tweets during the 2009 h1n1 outbreak travelling waves in the occurrence of dengue haemorrhagic fever in thailand detecting influenza epidemics using search engine query data rakai health sciences program. the role of viral introductions in sustaining community-based hiv epidemics in rural uganda: evidence from spatial clustering, phylogenetics, and egocentric transmission models middle east respiratory syndrome coronavirus in dromedary camels: an outbreak investigation the parable of google flu: traps in big data analysis wikipedia usage estimates prevalence of influenza-like illness in the united states in near real-time a molecular basis for the interplay between t cells, viral mutants, and human leukocyte antigen micropolymorphism assessing and maximizing the acceptability of global positioning system device use for studying the role of human movement in dengue virus transmission in iquitos predictive mapping of human risk for west nile virus (wnv) based on environmental and socioeconomic factors a high-resolution human contact network for infectious disease transmission revealing the microscale spatial signature of dengue transmission and immunity in an urban population the evolution of hiv-1 and the origin of where diseases and networks collide: lessons to be learnt from a study of the 2001 foot-and-mouth disease epidemic the use of twitter to track levels of disease activity and public concern in the u.s. during the influenza a h1n1 pandemic air travel and vectorborne disease movement usefulness of commercially available gps data-loggers for tracking human movement and exposure to dengue virus using gps technology to quantify human mobility, dynamic contacts and infectious disease dynamics in a resource-poor urban environment contrasting the epidemiological and evolutionary dynamics of influenza spatial transmission viral phylodynamics measles metapopulation dynamics: a gravity model for epidemiological coupling and dynamics key: cord-334751-7mdafd2y authors: mattson, stephanie l.; higbee, thomas s.; aguilar, juliana; nichols, beverly; campbell, vincent e.; nix, lyndsay d.; reinert, kassidy s.; peck, sara; lewis, kylee title: creating and sharing digital aba instructional activities: a practical tutorial date: 2020-07-23 journal: behav anal pract doi: 10.1007/s40617-020-00440-z sha: doc_id: 334751 cord_uid: 7mdafd2y board certified behavior analysts (bcbas) may encounter situations, such as the current covid-19 pandemic, that preclude them from providing traditional in-person applied behavior-analytic services to clients. when conditions prevent bcbas and behavior technicians from working directly with clients, digital instructional activities designed by bcbas and delivered via a computer or tablet may be a viable substitute. google applications, including google slides, google forms, and google classroom, can be particularly useful for creating and sharing digital instructional activities. in the current article, we provide task analyses for utilizing basic google slides functions, developing independent instructional activities, developing caregiver-supported instructional activities, and sharing activities with clients and caregivers. we also provide practical recommendations for implementing digital instructional activities with clients and caregivers. effective, but some situations, such as the current covid-19 global pandemic, may cause service disruptions. unfortunately, many individuals who receive aba services may struggle to maintain skills when long gaps without services occur (lotfizadeh, kazemi, pompa-craven, & eldevik, 2020) . given that aba-based treatments have been deemed medically necessary, it will be important for service providers to work with clients and families to provide appropriate alternative instructional activities when it is not feasible to conduct traditional in-person treatment. recently, researchers have investigated the efficacy of using telehealth to provide behavior-analytic services from a distance. previously, telehealth models of service delivery have been used in situations where there are geographic barriers to conducting in-person treatment (eid et al., 2017; fisher et al., 2014; suess et al., 2014) . prior researchers have demonstrated the efficacy of using telecommunication technology to train caregivers to assess and treat problem behavior (suess et al., 2014) and implement skill acquisition programming using dtt (eid et al., 2017) . although research indicates that caregivers can be effective implementers, it may be challenging for them to develop materials that can be utilized during instructional programming. moreover, digital instructional activities can have embedded schedules to provide structure for how and when instructional editor's note this manuscript is being published on a highly expedited basis, as part of a series of emergency publications designed to help practitioners of applied behavior analysis take immediate action to adjust to and mitigate the covid-19 crisis. this article was submitted on april 10, 2020, and received final acceptance on april 17, 2020. the journal would like to especially thank dr. richard colombo for his expeditious review of the manuscript. the views and strategies suggested by the articles in this series do not represent the positions of the association for behavior analysis international or springer nature. material is presented to the learner. one alternative may be to develop and disseminate digital instructional activities that parents can implement with their children. digital instructional activities are commonly used across a variety of instructional settings, and researchers have created tutorials for using accessible technology platforms to create these activities (e.g., blair & shawler, 2019; cummings & saunders, 2019) . there are several advantages to creating and sharing digital instructional content, particularly during the covid-19 global pandemic. first, digital content can be easily developed, edited, and rearranged by the bcba. this may alleviate some of the burden on caregivers to locate, organize, and arrange instructional materials and activities. second, digital platforms provide bcbas with the unique opportunity to program feedback into instructional activities. this digital feedback may allow clients to engage with the instructional programming with less prompting and support, which may be particularly valuable during the current crisis, as caregivers may be balancing work and homeschooling responsibilities for other children in the home. finally, caregivers may be able to use digital instructional content even beyond the current covid-19 crisis to supplement traditional instruction and mitigate inperson service interruptions that may occur due to unforeseen circumstances such as changes in geographic location, illness, or injury. in traditional in-person intervention, common instructional programs for children with autism spectrum disorder (asd) and related disorders are created to increase both listener and speaker responding through instructor-led learning trials in the home or clinic. we have found that google applications, such as google slides (google llc, 2006) and google forms (google llc, 2008) , can be particularly useful for creating digital versions of these types of instructional programs with which learners can interact, either independently or with the support of a caregiver, given their universal availability and functionality across multiple devices and operating systems. bcbas can use google slides to create a variety of activities in which a learner receptively identifies stimuli from an array (e.g., colors, shapes, numbers, letters, objects) or identifies parts of a single image (e.g., body parts, prepositions). using the google slides application, bcbas can make interactive programs that learners can complete independently by incorporating praise, prompting, and error correction into the digital instructional content that are delivered automatically as the learner interacts with the activity. it may also be useful to provide clients and families with caregiver-supported digital instructional content, which can be implemented by a caregiver with remote support from a bcba. bcbas can use google slides and google forms to create and organize many different caregiver-supported instructional activities, such as imitation, letter matching, sequencing, and expressive identification activities (e.g., identifying objects, answering personal information questions, answering wh-questions). after creating instructional content, bcbas will also need an easy way to organize and share learning activities with clients and families. google drive and google classroom applications are potential solutions, as they allow users to conveniently organize and share content with other users. bcbas can use google drive as a simple application to organize and share specific files, folders, and activities. google classroom-albeit a more complex solution to information and activity sharing-is another promising application that has a number of additional features that make it an attractive option for organizing behavior-analytic instructional activities and sharing them with clients and caregivers. google classroom, introduced in the google apps for education in 2014 (shaharanee, jamil, & rodzi, 2016) , allows teachers and clinicians to provide instruction, create assignments, and communicate with students through a digital platform (dicicco, 2016) . although research on the effects of google classroom instruction is limited, there have been some early positive results. dicicco found that seventh-grade social studies students with learning disabilities increased vocabulary scores after using direct instruction through google classroom. students also reported high levels of satisfaction related to using the google classroom platform (dicicco, 2016) . there are several practical advantages to using google applications to create and share instructional activities. first, all google applications, including google slides and google forms, can be seamlessly integrated into a google classroom. second, google applications are free and available on virtually all digital devices, making them a convenient technology platform for clients and families to access. third, google applications are used across many schools and workplaces, so it is possible that some clients and caregivers have experience using similar programs. in this article, we aim to provide a practical tutorial for using google applications to develop and disseminate behavior-analytic digital instructional activities. we will discuss and provide task analyses for the following: (a) utilizing basic functions within google slides (e.g., adding shapes, inserting images, linking stimuli, and protecting slides) to create interactive instructional materials, (b) developing independent instructional activities that learners can complete with minimal caregiver support, (c) developing caregiversupported instructional activities where the caregiver provides instruction using digital learning materials, and (d) organizing materials and sharing activities with clients and caregivers using google classroom. bcbas can create a variety of instructional activities using google slides, such as color identification, object identification, and activities that teach prepositions. one distinct advantage of using google slides is the ability to link learner responses to praise and error correction slides so that the learner automatically receives feedback appropriate to the response emitted. for example, consider a learner who is working on a color identification activity. the bcba who develops the activity can arrange the slides so that if the instruction is "touch red" and the learner touches red, the google slides presentation will automatically navigate the learner to a praise slide. if the learner touches green, the google slides presentation will automatically navigate the learner to an error correction slide. these types of activities are valuable for caregivers because they reduce some of the burden of prompting and error correction. in order to create the interactive digital content described previously, developers need to use several google slides functions. the following section includes task analyses that review the basic google slides component skills that bcbas will use to create instructional activities using the task analyses in subsequent sections. these component skills include (a) building a google slides presentation, (b) adding visual stimuli (e.g., shapes, images, and text boxes), (c) adding auditory stimuli, (d) adding links to stimuli, (e) adding navigation arrows, and (f) protecting the slide. 1. open a web browser and sign in to a google account. 2. navigate to http://docs.google.com/presentation. 3. create a new blank presentation by clicking the multicolored plus sign in the navigation bar near the top of the page. 4. create a title slide with the name of the instructional activity. 5. create a new blank page by clicking on the last page in the left column and pressing the "enter" key. 6. add desired content. 7. preview slides by selecting "present" at the top right of the screen (fig. 1 ). bcbas can create instructional stimuli by adding visual stimuli (e.g., shapes, images, and text boxes) to a google slides presentation. 1. select the shape icon on the toolbar (fig. 2) . 2. select a specific shape (fig. 2) . 3. click anywhere on the slide and drag the cursor to size the shape (fig. 3 ). a. holding the "shift" key while dragging the cursor will keep the shape square. doing so will make a perfect square, circle, or equilateral triangle (fig. 4) . 4. to change the color of the shape: a. select the shape. b. as shown in fig. 5 , select the fill color icon on the toolbar and select a specific color. c. as shown in fig. 6 , select the border color icon on the toolbar and select "transparent" or a specific color for the border. 1. as shown in fig. 7 , select the insert image icon on the toolbar and select an option for inserting the image. a. to insert an image using the "upload from computer" option, navigate the menu to where the image is saved, select the image, and select "open." b. to insert an image using the "search the web" option, enter an image type into the search box, select an image, and click "insert" (fig. 8 ). c. to insert an image using the "drive" or "photos" options, select an image from google drive or google photos and select "insert." d. to insert an image using the "by url" option, rightclick on an image from the internet. select "copy image address" (fig. 9 ), select the "by url" option, paste it into the field (fig. 10) , then select "insert." adding text boxes 1. as shown in fig. 11 , select the text box icon on the toolbar. 2. add the text box to the slide by clicking a specific location. drag with the cursor until the text box is the desired size. google slides allows users to add audio file content to slides. with this feature, bcbas can add vocal instructions, praise, a. under "start playing," select "automatically" so the audio starts without clicking on anything ( fig. 15 ). b. check "hide icon while presenting" so the icon is not visible or distracting to the learner (fig. 15 ). 6. select "present" (fig. 1) to verify that the audio plays automatically and that the icon is hidden on each slide. a. if a slide does not play the audio automatically, return to step 5a. b. if the icon is still visible, return to step 5b. bcbas can create an interactive experience for the learner by adding links to stimuli. linked stimuli can automatically navigate learners to a praise slide or error correction slide, back to the original instruction slide, or to subsequent instruction slides within the activity, depending on the learner's response. 1. select the shape or image on the slide to add a link to. fig. 16 , click the insert link icon on the toolbar. if the insert link icon is not visible, try closing the side bars. 3. select the "slides in this presentation" option ( fig. 17 ). 4. select which slide the shape/image should navigate to (i.e., praise slide or error correction slide). 5. select "apply" so the shape/image will navigate to the correct slide (fig. 18 ). 6. to ensure that the shape/image is connected to a link, select the shape/image and check for the white link bar below the shape/image (fig. 19 ). if the shape/image is not linked or linked to the wrong slide, select "remove link" and return to steps 1-5. navigation arrows allow the learner to easily navigate across the different slides within the instructional activity. adding navigation arrows 1. select the shape icon and select the right arrow (fig. 20) . draw in the bottom-right corner of the page. 2. double-click on the arrow and type "next" in the center of the arrow. resize and center as shown in fig. 21 . 3. repeat steps 1-2 to create a left arrow that says "go back." 4. place the "next" arrow in the bottom-right corner and "go back" in the bottom-left corner (fig. 22) . fig. 23 , link the arrows to navigate the learner to the appropriate slide (see the "adding links" section). select the shape and not the font when creating the link. 6. select "present" (fig. 1 ) and use the navigation arrows to progress through the slides. a. if a navigation arrow does not work correctly, check the link for that arrow by selecting the arrow and checking the white link bar below it (fig. 23 ). i. if the arrow is not linked or linked to the wrong slide, select "remove link" and follow steps 1-5 of "adding links." once all stimuli and links have been added, the bcba needs to add a final protective film to the slide to prevent the learner adding a protective film 1. as shown in fig. 24 , insert a rectangle that covers the entire slide (see the "adding shapes" section). typically, the rectangle will have a fill color. it is fine to leave the fill color for now. it will be adjusted later. this rectangle will be the protective film. 2. select the protective film and link it to the current slide (see "adding links"). this will prevent any clicks from transitioning to another slide. 3. as shown in fig. 25 , right-click the protective film, navigate to "order," and select "send to back." 4. select any nonlinked stimuli (e.g., instruction text box) and "send to back" as well so the only stimuli in front of the protective film are linked stimuli ( fig. 26 ; e.g., instructional stimuli, arrows). this will disable all clicks or presses anywhere on the page other than clicking or touching the stimuli. 5. for finishing touches, select the protective film and click the fill color icon (fig. 27) and border color icon (fig. 28 ) to change both colors to "transparent." 6. now the only selection options on the instructional slides should be the target response and distractor responses that will navigate the learner to the corresponding feedback. for praise and error correction slides (depending on the prompting strategy used), the navigation arrows should be the only selection options. to ensure that everything works as it should, use the "present" feature (see fig. 1 ) a. if the slide progresses to the next slide without the user selecting linked stimuli or a navigation arrow, ensure that the protective film is correctly linked by checking the white link bar below it. i. if the protective film is not linked or linked to a slide other than the current slide, select "remove link" and follow steps 1-5 of the "adding links" section to link the protective film to the current slide. to develop independent activities, bcbas can program praise, prompting, and error correction into the digital instructional activity (see fig. 65 in the appendix). digital activities that are designed to be completed by learners with minimal caregiver support could involve the learner either selecting a correct target response from an array of stimuli or selecting parts of a stationary picture. in the following sections, we have provided task analyses for both types of activities. in this type of activity, the learner selects one picture from an array. specific examples of instructional activities in which the learner responds by selecting from an array include receptive color, shape, object, and letter identification activities. creating an array of three 1. open a new presentation in google slides. 2. create a title slide. 3. create three blank slides for the first target response (instruction, praise, and error correction slides). when adding the links (in a future step), it may be helpful to title these three slides accordingly (1, instruction; 2, praise, and 3, error correction). 4. add the instruction for the target to the first instruction slide in the slide title area or by adding a text box (see the "adding text boxes" section). 5. delete extra text boxes on the slide. 6. insert the target stimulus and two distractor stimuli (see "adding shapes" and "inserting images"). 7. as shown in fig. 29 , use the guide bars to line up the three stimuli in a row. 8. add a link for the correct stimulus to navigate the learner to a praise slide (see "adding links"). 9. add links for the distractor stimuli to navigate the learner to the error correction slide (see "adding links"). 10. add a vocal instruction to the slide (see "inserting audio files"). 11. add the protective film to the slide (see "adding a protective film"). 12. navigate to the praise slide and add content (see "practice recommendations," later in the article). a few options include: a. textual praise statements (see "adding text boxes"). b. auditory praise statements (see "inserting audio files"). pictures (see "inserting images"). 13. add a navigation arrow to advance to the next instructional slide (see "adding navigation arrows"). 14. add the protective film to the praise slide (see "adding a protective film"). 15. navigate to the error correction slide and add content (see "practice recommendations," later in the article). a few options include: a. textual error correction statements (see "adding text boxes"). b. auditory error correction statements (see "inserting audio files"). c. pictures (see "inserting images"). d. prompts (see "practice recommendations"). e. add navigation arrow to either go back to the previous instructional slide or link the stimulus (see "adding navigation arrows"). 16. add the protective film to the error correction slide (see "adding a protective film"). 17. repeat steps 2-15 to build the remainder of the instructional activity. 18. double-check that everything is working as it should by using the "present" feature ( fig. 1) . a. if audio does not begin playing automatically or the icon is visible, refer to the troubleshooting tips in step 6 of the "inserting audio files" section. b. if stimuli are not navigating to the correct slides, refer to the troubleshooting tips in step 6 of "adding links." c. if navigation arrows are not navigating to the correct slides, refer to the troubleshooting tips in step 6 of "adding navigation arrows." d. if slides are progressing by clicking anywhere on the slide that is not a linked stimulus or navigation arrow, refer to the troubleshooting tips in step 6 of "adding a protective film." in this type of activity, the learner selects part of a stationary picture. specific examples of instructional activities in which the learner responds by selecting part of a stationary picture include receptive body-part identification and preposition activities. creating a stationary background 1. open a new presentation in google slides. 2. create a title slide. 3. create three blank slides for the first target response (instruction, praise, and error correction slides). when adding the links (in a future step), it may be helpful to title these three slides accordingly (1, instruction; 2, praise; and 3, error correction). 4. paste or insert the image to the instruction slide (see "inserting images"). once the stationary background is created, it can be turned into an interactive activity by adding shapes that are linked to praise and/or error correction slides. creating interactive activities with a stationary background 1. add a textual stimulus for the current target (e.g., "touch head"; see "adding text boxes"). 2. add an auditory stimulus for the current target (e.g., "touch head"; see "inserting audio files"). 3. insert a shape that is appropriate for the section of the stationary image that is considered the correct response (see "adding shapes"). fig. 34 , place the shape on top of the area of the stationary picture that is the target response. for this example, a circle was added around the "head" portion of the stationary background. 5. add a link to the shape that will navigate the learner to the praise slide (see "adding links"). 6. add the protective film to the slide, leaving only the target response shape above the film (see "adding a protective film"). please note that the protective film in this activity/ program should be linked to the error correction page. thus, in this example, any selection outside of "head" will be considered an incorrect response. 7. make the fill color (fig. 27 ) and border color (fig. 28 ) of the shape covering the target response transparent. 8. add in praise and error correction slides. 9. double-check that everything is working as it should by using the "present" feature ( fig. 1) . to the troubleshooting tips in step 6 of "adding links." c. if navigation arrows are not navigating to the correct slides, refer to the troubleshooting tips in step 6 of "adding navigation arrows." d. if slides are progressing by clicking anywhere on the slide that is not linked stimuli or a navigation arrow, refer to the troubleshooting tips in step 6 of "adding a protective film." in order to complete caregiver-supported instructional activities, the learner will need support from a caregiver who will facilitate the instructional activity and provide praise and error correction (if necessary) following learner responses. there are multiple ways to develop caregiver-supported instructional activities. bcbas can (a) use basic google slides functions to arrange instructional content into a google slides presentation (see the previous section "basic google slides functions"), (b) use google slides to develop activities where learners drag and drop stimuli on a slide (approximating activities in which learners in this type of activity, the learner drags items to a specific location on the slide and a caregiver provides feedback on responses. specific examples of drag-and-drop instructional activities include letter matching, sequencing, patterning, and sorting activities. creating drag-and-drop activities. 2. create the stationary components for the drag-and-drop activity. the following example will provide instructions for creating a letter matching activity. these steps can be used to create a variety of drag-and-drop activities. a. insert a text box (see "adding text boxes") and type the instructions (fig. 35) . b. insert the desired shape (see "adding shapes"; fig. 36 ). c. add additional stimuli as needed for the instructional activity (see "adding shapes" and "inserting images" ; fig. 37 ). 3. create a stationary background (see "creating a stationary background"). fig. 38 , insert mobile objects with individual text boxes (see "adding text boxes") and/or shapes (see "adding shapes"). do not add letters in the text boxes. mobile objects need to be text boxes with shapes or pictures. 5. share the activity with clients by selecting the "share" button and entering the client's e-mail (fig. 39) . clients can interact with the activity in the slide, and the caregiver can provide feedback. please note that slides are not mobile in "present" mode. in this type of activity, the caregiver presents the question listed in the google form, the learner responds, and the caregiver provides feedback and marks whether the learner answered correctly or incorrectly. specific examples of instructional activities bcbas can arrange in a google form include nonverbal imitation, speaker responding (e.g., shapes, colors, objects, letters), and personal information question activities. 1. navigate to google forms by entering https://www. google.com/forms/about/ into a web browser. 2. click on "go to google forms" (see fig. 40 ). 3. click the "blank" form with the multicolored plus sign as shown in fig. 41 fig. 42 , click on "untitled form" in the upper left corner of the screen and type the title (e.g., "personal information") of the instructional activity. 5. click the description box and enter the instructions for conducting the instructional activity (fig. 43 ). 6. navigate to "untitled question" below the form's "title" box. 7. create a form identifier by instructing users to identify who is conducting the session (e.g., "enter your id") and selecting "short answer" from the drop-down menu (fig. 44) . creating a form identifier will allow those viewing the responses to easily organize form responses (see "viewing form responses" later in the article). 8. click the "required" button on the bottom of the screen (fig. 44) . 9. click the add question icon (fig. 45) . 10. select the "question" box and enter a question into the box (e.g., "what's your first name?"). 11. from the drop-down menu, select "multiple choice" (fig. 46) . 12. click "option 1" and type "answered correctly" (fig. 47) . 13. click "add option" and type "answered incorrectly" (fig. 47) . 14. click the add question icon (fig. 45 ) and repeat steps 10-13. viewing form responses 1. navigate to google forms by entering https://www. google.com/forms/about/ into a web browser. 2. click on "go to google forms" to view quizzes (see fig. 40 ). 3. select a quiz. 4. along the top of the selected quiz, click on the "responses" button (fig. 48) . 5. the response page will allow the reader to view the following: a. to view a summary of all responses, click on the "summary" button (fig. 49) . b. to view a summary of each question, click on the "question" button (fig. 49 ). c. to view a summary of each individual quiz, click on the "individual" button (fig. 49 ). sheets, click on the google sheets icon. there are several options for sharing instructional materials with clients and families using the google drive and google classroom applications. bcbas can create and share a link to an individual google drive file, create and share a google drive folder, or build a google classroom to organize and store multiple activities. it is important to note the following: 1. bcbas should publish any google slides instructional activities that have linked slides (e.g., selecting colors, numbers, or letters from an array; identifying body parts from a stationary background). sharing an individual file 1. navigate to https://drive.google.com. open the file to share. 3. select the yellow "share" button in the top-right corner. 4. click on "get shareable link." 5. a pop-up menu will appear that provides sharing options (fig. 50) : a. select "anyone with the link can view" or "anyone with the link can edit." b. select "copy link." 6. share the link with the family/client by pasting it into a message. creating and sharing a folder 1. navigate to https://drive.google.com. 2. \select the multicolored plus sign on the top-left side of the page. 3. select "folder" in the drop-down menu. 4. name the new folder. 5. add items/documents to the folder by dragging and dropping files or clicking the "new" button. 6. right-click on the folder and select the "share" option ( fig. 51) . publishing slides 1. open a completed google slides presentation. 2. navigate to "file" located on the toolbar. 3. from the drop-down menu, select "publish to the web" (fig. 52) . 4. a "publish to the web" pop-up menu will appear (fig. 53 ). a. in the "auto-advance slides" drop-down menu, choose "every minute." b. check the box next to "start slideshow as soon as the player loads." c. click "publish." 5. right-click on the link. copy and share the link with the family/client by pasting it into a message. to set up a google classroom, bcbas will need a gmail account. for privacy purposes, we recommend creating an individual classroom for each client. it is important to note that only individuals with gmail accounts can join and interact with the classroom. 1. navigate to https://classroom.google.com/h. 2. select the "+" in the top-right corner to create a new classroom (see fig. 54 ). 3. select "create class" from the drop-down menu. 4. a dialogue box will appear inquiring as to whether or not the classroom will be utilized for a school. if you are using google classroom as a private agency, check the box and click continue (see fig. 55 ). 5. type the class name into the dialogue box (fig. 56) . 6. type a section name (e.g., client's initials) in the following tab (fig. 56) . 7. select "create" (see fig. 56 ). 8. the classroom will now appear in the google classroom homepage (see fig. 57 ). 1. provide clients and/or caregivers with the google classroom code (see fig. 58 ). 2. direct clients and/or caregivers to navigate to https:// classroom.google.com/h. 3. clients and caregivers will need to select the "+" in the top-right corner (fig. 54) , click "join class," enter the class code, and click "join." (see fig. 59 ). google classroom is a digital teaching platform that allows instructors to organize instructional content by topic, add assignments, and post announcements. by following the process in the next sections, bcbas can add any digital content developed using google slides or google forms into a google classroom assignment. 1. navigate to the "classwork" tab. 2. click "create" and select "topic" from the drop-down menu (see fig. 60 ). fig. 61 , type a topic name (e.g., "independent activities") into the box, and click "add." drag and drop instructional content into the topic area from the assignment list above or assign a topic while creating an assignment (see the next section, "creating assignments"). 1. navigate to the "classwork" tab. 2. click "create" and select "assignment" from the dropdown menu (see fig. 62 ). 3. type an assignment title in the "title" text box and provide instructions for clients/parents in the "instructions" field, if necessary (fig. 63) . 4. attach a google slides activity, google form, or other materials by clicking the "add" button (fig. 63) . a. to insert instructional content from google drive, select "google drive," navigate to the relevant file, and click "add." b. to insert a link, select "link," copy and paste the link into the box, and click "add link." c. to insert a computer file, select "file," drag the file into the box or select it from a device, and click "upload." 5. adjust points, due date, topic, and rubric as necessary by clicking on the respective drop-down menus, as shown in fig. 64 . 6. navigate to the top-right corner of the screen and click the arrow next to the blue "assign" button. select "assign," "schedule," or "save as draft." 7. repeat steps 2-6 to develop additional assignments. although the google applications described previously provide an online platform for creating and sharing digital instructional activities, it is important to note that bcbas will still need to assess clients' and caregivers' prerequisite skills, train caregivers to conduct the activity, structure the instructional trial arrangement, program feedback, and develop a data collection system. bcbas should assess clients' and caregivers' prerequisite skills prior to implementing digital instructional activities. in order to effectively interact with independent instructional content, clients need basic computer skills (e.g., ability to navigate a keyboard or touch screen), attending and discrimination skills (e.g., ability to attend to, scan, and select stimuli on a screen), and basic independent work skills (e.g., ability to engage with the instructional content for the duration of the activity with minimal caregiver support). before assigning caregiver-supported instructional activities, bcbas should consider the prerequisite skills clients need to successfully participate in each activity. bcbas should also assess caregivers' prerequisite skills (e.g., stimulus/instructional control) and provide appropriate training, perhaps via telehealth, to ensure that the caregiver can facilitate client interaction with digital instructional activities effectively. there are many ways to provide telehealth support using videoconferencing platforms such as zoom (zoom video communications, inc., 2011 ), vsee (vsee lab, inc, 2008 ), or gotomeeting (logmein, 2004 ). an in-depth guide to parent training is beyond the scope of this article. however, there are behav analysis practice several behavior-analytic training strategies that may be useful, including behavioral skills training (sarakoff & sturmey, 2004) and remote in vivo coaching (bowles & nelson, 1976) . bcbas should also be aware that there are many ways to arrange trials within instructional activities. common options include massed-trial teaching (mtt) and interspersed-trial teaching (itt). during traditional mtt, an instructor presents the same acquisition target several times consecutively within a session (henrickson, rapp, & ashbeck, 2015) . during itt, an instructor incorporates mastered and/or nonmastered targets within a session (henrickson et al., 2015) . the google slides and google forms applications allow bcbas to design an instructional sequence in many different ways. thus, when creating digital instructional activities, bcbas should continue to customize and individualize client programming. praise research has demonstrated that praise may function as a reinforcer for some children with asd and other related disorders (joachim & carroll, 2018; paden & kodak, 2015) . bcbas can use google slides' functions to incorporate praise into independent instructional activities by creating praise slides and linking them to the "correct answer" stimuli. bcbas should customize praise slides to an individual learner's preferences. for example, praise slides may include prerecorded auditory praise-statement stimuli (e.g., "good job!") or sound clips (e.g., bells, dings, favorite songs). praise slides can also include visual stimuli such as cartoon images (e.g., favorite characters, balloons, or flowers), moving images (e.g., .gif files), or videos (e.g., youtube links). bcbas will also need to train caregivers to provide praise and reinforcement during caregiver-supported instructional activities. prompting and error correction when completing instructional activities, clients are likely to make errors. therefore, bcbas need to consider the types of error correction procedures that will be most appropriate for their clients (leaf et al., 2016) . common components of an error correction procedure include (a) demonstration of the correct response, (b) an active client response (requiring a student response after the prompt), (c) repeated representation of the instruction, and (d) differential reinforcement of the correct response (cariveau, la cruz montilla, ball, & gonzalez, 2019) . there are multiple ways to build error correction into independent instructional activities using google slides. for example, bcbas can provide prompts on "error correction" slides. prompts may include the addition of pictures that point to the correct answer (e.g., a finger or an arrow), animations of the correct answer (e.g., making the correct answer spin or zoom in and out), or the addition of an auditory prompt that includes the correct response (e.g., "this is the square."). the error correction slide can include a simple visual/auditory prompt, or the bcba can elect to include a link among the possible responses to require an active student response. bcbas will also need to consider prompting strategies for caregiver-supported activities (e.g., providing a vocal prompt of the correct response, or requiring an independent response before continuing) and provide caregivers with the proper training to correct errors. data collection is an important part of any aba program (baer, wolf, & risley, 1968) , and it is particularly important during interruptions to in-person services. it may be possible for the bcba to record instructional sessions (provided that he or she has consent from the caregiver) and collect data from the recording. if this type of data collection is not possible, it is important to design systems that caregivers can easily manage in the home. existing literature suggests that estimation (ferguson et al., 2019; taubman, leaf, mceachin, papovich, & leaf, 2013) and probe (ferguson et al., 2019; lerman, dittlinger, fentress, & lanagan, 2011 ) data collection systems can provide a reasonable estimate of skill acquisition performance. caregivers can also use google forms to collect data on instructional activities. as demonstrated previously (see "creating a google form"), bcbas can arrange google forms so that the caregiver or family member running the activity can quickly indicate whether the client answered correctly or incorrectly. once the google form is submitted, these data can be organized and analyzed easily. during service disruptions such as those caused by the current covid-19 global pandemic, bcbas may need to consider alternative ways to provide clients with appropriate instructional activities. in today's society, there are many technologybased options for creating digital activities to mitigate service gaps in situations where in-person treatment is not feasible. google applications, such as google slides, google forms, and google classroom, provide bcbas with an easy and accessible way to create and share these digital materials. although there are many potential benefits of creating digital instructional activities using google applications and planning for their use with clients, there are some potential limitations worth mentioning. first, recipients need to have a gmail address to access and use the materials. second, bcbas may need to carefully check and troubleshoot issues with linking stimuli, audio files, navigation arrows, and protecting the slide. we have incorporated troubleshooting suggestions into the task analyses for these particular functions. finally, creating high-quality digital instructional content can be a time-consuming process. given that clients and families will likely have immediate service needs, bcbas will need to find other ways to continue treatment, perhaps by conducting standard instructional programming via telehealth, while simultaneously preparing digital instructional activities to supplement these telehealth-based services. we hope that this article will provide bcbas with some useful tools for creating digital content to share with clients and caregivers. although there is no question that direct intervention by trained behavior technicians, under the supervision fig. 64 assignment-specific adjustments fig. 63 adding content to an assignment of a behavior analyst, would likely produce significantly more beneficial impacts than interacting with the activities proposed in this article, it is possible that interaction with these instructional activities may mitigate some of the skill loss that could occur due to a disruption in direct service provision. future researchers may address this question. in addition to potentially reducing the impact of service disruptions during the current crisis, bcbas may be able to provide digital content to support clients when there are geographic barriers to inperson treatment or when unforeseen circumstances (e.g., illness or injury to a family member) arise. when standard aba services resume, bcbas may also incorporate digital instructional activities to supplement in-person services. as such, we hope that the processes described previously will be useful both during and beyond the covid-19 pandemic. conflict of interest the authors have no known conflicts of interest to disclose. informed consent as this is a technical article, no human participants were involved in the project. thus, no informed consent was necessary. online voice recorder (version 1.1) [web software some current dimensions of applied behavior analysis developing and implementing emergent responding training systems with available and low-cost computer-based learning tools: some best practices and a tutorial training teachers as mediators: efficacy of a workshop versus the bug-in-the-ear technique a preliminary analysis of equivalence-based instruction to train instructors to implement discrete trial teaching naturalistic teaching strategies (nats) to teach speech to children with autism: historical perspective, development, and current practice using powerpoint 2016 to create individualized matching to sample sessions the effects of google classroom on teaching social studies for students with learning disabilities training parents in saudi arabia to implement discrete trial teaching with their children with autism spectrum disorder an evaluation of estimation data collection to trial-by-trial data collection during discrete trial teaching preliminary findings of randomized clinical trial of a virtual training program for applied behavior analysis technicians google slides [web application google forms [web application teaching with massed versus interspersed trials: effects on acquisition, maintenance, and problem behavior a comparison of consequences for correct responses during discrete-trial instruction an evaluation of positional prompts for teaching receptive identification to individuals diagnosed with autism spectrum disorder a comparison of methods for collecting data on performance during discrete trial teaching gotomeeting [web software moderate effects of low-intensity behavioral intervention teaching developmentally disabled children: the me book voice recorder [computer software the effect of reinforcement magnitude on skill acquisition with children with autism the effects of behavioral skills training on staff implementation of discrete-trial teaching google classroom as a tool for active learning evaluating the treatment fidelity of parents who conduct in-home functional communication training with coaching via telehealth a comparison of data collection techniques used with discrete trial teaching vsee clinic (version 4.0) [computer software publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations key: cord-310769-y6orh217 authors: zaman, a.; zhang, b.; hoque, e.; silenzio, v.; kautz, h. title: the relationship between deteriorating mental health conditions and longitudinal behavioral changes in google and youtube usages among college students in the united states during covid-19: observational study date: 2020-08-25 journal: nan doi: 10.1101/2020.08.22.20178640 sha: doc_id: 310769 cord_uid: y6orh217 background: mental health problems among the global population are worsened during the coronavirus disease (covid-19). yet, current methods for screening mental health issues rely on in-person interviews, which can be expensive, time-consuming, blocked by social stigmas and quarantines. meanwhile, how individuals engage with online platforms such as google search and youtube undergoes drastic shifts due to covid-19 and subsequent lockdowns. such ubiquitous daily behaviors on online platforms have the potential to capture and correlate with clinically alarming deteriorations in mental health profiles of users through a non-invasive manner. objective: the goal of this study is to examine, among college students in the united states, the relationship between deteriorating mental health conditions and changes in user behaviors when engaging with google search and youtube during covid-19. methods: this study recruited a cohort of undergraduate students (n=49) from a u.s. college campus during january 2020 (prior to the pandemic) and measured the anxiety and depression levels of each participant. the anxiety level was assessed via the general anxiety disorder-7 (gad-7). the depression level was assessed via the patient health questionnaire-9 (phq-9). this study followed up with the same cohort during may 2020 (during the pandemic), and the anxiety and depression levels were assessed again. the longitudinal google search and youtube history data of all participants were anonymized and collected. from individual-level google search and youtube histories, we developed 5 signals that can quantify shifts in online behaviors during the pandemic. we then assessed the differences between groups with and without deteriorating mental health profiles in terms of these features. results: of the 49 participants, 41% (n=20) of them reported a significant increase (increase in the phq-9 score > 5) in depression, denoted as dep; 45% (n=22) of them reported a significant increase (increase in the gad-7 score > 5) in anxiety, denoted as anx. of the 5 features proposed to quantify online behavior changes, statistical significances were found between the dep and non-dep groups for all of them (p<.01, effect sizes eta_{partial}^2 ranging between 0.130 to 0.320); statistical significances were found between the anx and non-anx groups for 4 of them (p<.02, effect sizes eta_{partial}^2 ranging between 0.115 to 0.231). significant features included late-night online activities, continuous usages and time away from the internet, porn consumptions, and keywords associated with negative emotions, social activities, and personal affairs. conclusions: the results suggested strong discrepancies between college student groups with and without deteriorating mental health conditions in terms of behavioral changes in google search and youtube usages during the covid-19. though further studies are required, our results demonstrated the feasibility of utilizing pervasive online data to establish non-invasive surveillance systems for mental health conditions that bypasses many disadvantages of existing screening methods. globally, mental health problems such as depression, anxiety, and suicide ideations are severely worsened during the coronavirus disease (covid-19) [1] [2] [3] , specifically for college students [ 4 ]. yet, current methods for screening mental health issues and identifying vulnerable individuals rely on in-person interviews. such assessments can be expensive, time-consuming, and blocked by social stigmas, not to mention the reluctancy induced by travel restrictions and exposure risks. it has been reported that very few patients in need were correctly identified and received proper mental health treatments on time under the current healthcare system [ 5, 6 ] . even with emerging telehealth technologies and online surveys, the screening requires patients to actively reach out to care providers. at the same time, because of the lockdown enforced by the global pandemic outbreak, people's engagements with online platforms underwent notable changes, particularly in search engines trends [7] [8] [9] ] and exposures to media reports [ 10, 11 ]. reliance on internet has significantly increased due to the overnight change in lifestyles, for example, working and remote learning, imposed by the pandemic on the society. the sorts of content consumed, the time and duration spent online, and the purpose of online engagements may be influenced by covid-19. furthermore, the digital footprints left by online interactions may reveal information about these changes in user behaviors. most importantly, such ubiquitous online footprints may provide useful signals of deteriorating mental health profiles of users during covid-19. they may capture insights of what was going on in the mind of the user through a non-invasive manner, especially since google and youtube searches are short and succinct and can be quite rich in providing the in the moment cognitive state of a person. on one hand, online engagements can cause fluctuations in mental health. on the other hand, having certain mental health conditions can cause certain types of online behaviors. this opens up possibilities for potential healthcare frameworks which leverage pervasive computing approaches to monitor mental health conditions and deliver interventions on-time. extensive researches have been conducted on a population level, correlating mental health problems with user behaviors on social platforms [ 12, 13 ] , especially among young adolescents. researchers monitored twitter to understand mental health profiles of the general population such as suicide ideations [ 14 ] and depressions [ 15 ] . similar researches have been done with reddit, where anxiety [ 16 ] , suicide ideations [ 14 ] , and other general disorders were studied [ 17, 18 ] . another popular public platform is facebook, and experiments have been done studying anxiety, depression, body shaming, and stress online [ 19, 20 ] . however, such studies were limited to macro observations and failed to identify individuals in need of mental health assistances. in addition, it has been shown that college student communities rely heavily on youtube for both academic and entertainment purposes [ 21, 22 ]. yet, abundant usages may lead to compulsive youtube engagements [ 23 ] , and researchers have found that social anxiety is associated with youtube consumptions in a complex way [ 24 ] . during covid-19, multiple studies have reported deteriorating mental health conditions in various communities [1] [2] [3] 25 ], such as nation-wise [ 26, 27 ] , across the healthcare industry [ 28, 29 ] , and among existing mental health patients [ 30 ] . besides, online behaviors during covid-19 have been explored, especially for web searches related to the pandemic [ 7-9 ] and abnormal tv consumptions during the lockdown [ 10 ] . many of the behavioral studies also discussed about the effects of online interactions on the spread, misinformation, knowledge, and protective measures of covid-19, including the roles of youtube [ [31] [32] [33] ] and other platforms [ 34 ] . [ 35 ] investigated hate speech targeting the chinese and asian communities on twitter during covid-19. ubiquitous data has been proved to be useful in detecting mental health conditions. mobile sensor data, such as gps logs [ 36, 37 ] , electrodermal activity, sleep behavior, motion, and phone usage patterns [ 38 ] has been applied in investigating depressive symptoms. [ 39 ] found that individual private google search histories can be used to detect low self-esteem conditions among college students. it has been shown that online platforms preserve useful information about the mental health conditions of users, and covid-19 is jeopardizing the mental wellbeing of the global community. thus, we demonstrate the richness of online engagements logs and how it can be leveraged to uncover alarming mental health conditions during covid-19. in this study, we aim to examine whether the changes in user behaviors during covid-19 have a relationship with deteriorating mental health profiles. we focus on google search and youtube usages, and we investigate if the behavior shifts when engaging with these two platforms signify worsened mental health conditions. we hypothesize that late night activities, compulsive and continuous usages, time away from online platforms, porn and news consumptions, and key words related to health, social engagements, personal affairs, and negative emotions may play a role in deteriorating mental health conditions. the scope of the study covers undergraduate students in the u.s. we envision this project as a pilot study: it may lay a foundation for mental health surveillance and help delivery frameworks based on pervasive computing and ubiquitous online data. comparing to traditional interviews and surveys, such non-invasive system may be cheaper, efficient, and avoid being blocked by social stigmas while notifies the caregivers on-time about individuals at risk. we recruited a cohort of undergraduate students, all of whom were at least 18 years old and have an active google account for at least 2 year, from the university of rochester river campus, rochester, ny, u.s.a. the participation was voluntary, and individuals had the option to opt out of the study at any time, although we did not encounter any such cases. we collected individual-level longitudinal online data (google search and youtube) in the form of private history logs from the participants. for every participant, we measured the depression and anxiety levels via the clinically validated patient health questionnaire-9 (phq-9) and generalized anxiety disorder-7 (gad-7), respectively. basic demographic information was also recorded. there were in total two rounds of data collection: the first round during january 2020 (prior to the pandemic) and the second round during may 2020 (during the pandemic). during each round, for each participant, the anxiety and depression scores were assessed, and the change in mental health conditions were calculated in the end. the entire individual online history data up till the date of participation was also collected in both rounds from the participants. figure 2 gives an illustration of the recruitment timeline and two rounds of data collections. all individuals participated in both rounds and were compensated with 10-dollar amazon gift cards during each round of participation. given the sensitivity and proprietary nature of private google search and youtube histories, we leveraged the google takeout web interface [ 40 ] to share the data with the research team. prior to any data cleaning and analysis, all sensitive information such as the name, email, phone number, social security number, and credit card information was automatically removed via the data loss prevention (dlp) api [ 41 ] of google cloud. for online data and survey response storage, we utilized a hipaacompliant cloud-based secure storing pipeline. the whole study design, pipelines, and survey measurements involved were similar to our previous setup in [ 39 ] and have been approved by the institutional review board (irb) of the university of rochester. the google takeout platform enables users to share the entire private history logs associated with their google accounts, and as long as the account of the user was logged in, all histories would be recorded regardless of which device the individual was using. each activity in google search and youtube engagement logs were timestamped, signifying when the activity happened to the precision of seconds. besides, for each google search, the history log contained the query text input by the user. it also recorded the url if the user directly input a website address to the search engine. for each youtube video watched by the user, the history log contained the url to the video. if the individual directly searched with key word(s) on the youtube platform, the history log also recorded the url to the search results. in order to capture the change in online behaviors for the participants, we first introduced a set of features that quantifies certain aspects of how individuals interact with google search and youtube. the set of features was calculated for each participant separately. individual-level behavior changes were then obtained by examining the features variations between january to mid-march of 2020 (prior to the outbreak) and mid-march to may of 2020 (after the outbreak). concretely, we defined 5 features and cut the longitudinal data of each participant into two segments by mid-march, around the time of the covid-19 outbreak in the u.s and campus lockdown. the two segments spanned 2.5 months before and after mid-march, respectively, and data before january 2020 was discarded. the same feature was extracted from both segments of data, and the change was calculated. such change was referred as the behavior shifts during the pandemic and lockdown. figure 2 gives an illustration of data segmentations and feature engineering pipelines. we defined late night activities (lna) as the activities happened between 10:00 p.m. and 5:00 a.m. of the next day, regardless of google search or youtube. for each participant, we counted the numbers of late-night activities before ( ()*+#) ) and after the outbreak ( ()*+#) ), respectively. we then calculated the percentage change of late-night activities and used it as a behavior shift feature: . the percentage change of late nigh activities before and after covid-19 outbreak. for the rest of the study, any mentioned percentage or relative changes of features were calculated the same way as above. we defined inactivity periods as the periods of time where no google search nor youtube activity was performed. we set a threshold of 7 hours, and we identified all the inactivity periods that were longer than 7 hours for each participant from the online data log. moreover, we looked at how these inactivity periods were distributed across 24 hours. we obtained the mid-point hour mark for each inactivity period: for example, an inactivity period started at 11 p.m. and ended at 7 a.m. has a mid-point of 3 a.m. with normalization, we received a discrete distribution of inactivity period midpoints over the 24-hour bins. it represented how the time away from google search and youtube of an individual was distributed in a 24-hour period. such distribution was calculated on the data segments before ( ()*+#) ) and after ( "*$)# ) the outbreak, respectively. figure 1 showcases two normalized inactivity midpoint distributions before and after the outbreak: after the outbreak, most of the inactive periods of participant 1 shifted to later hours of the dawn, which was most likely to be a delay in bedtime; for participant 2, the morning inactivity moved earlier, and new inactive periods during the afternoon appeared after the outbreak. one possible explanation could be that participant 2 started to take naps after the noon, resulting in midpoints around 5 p.m. all rights reserved. no reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted august 25, 2020. . to estimate the difference before and after the outbreak, we calculated the kldivergence [ 42 ] between the two distributions for each participant: equation 2 . the kl divergence of inactivity distributions before and after covid-19 outbreak. the kl-divergence is strictly greater than or equals to 0, and it equals to 0 only when the two distributions are identical. we defined short event interval (sei) as the period of time that is less than 5 minutes between two adjacent events. it usually occurs when one is consuming several youtube videos or searching for related contents in a roll. we counted the total numbers of short event intervals for each participant before ( ()*+#) ) and after ( "*$)# ) the outbreak, respectively. we calculated the percentage change of sei the same way as equation 1 and used it as a behavioral feature. the linguistic inquiry and word count (liwc) is a toolkit used to analyze various emotions, cognitive processes, social concerns, and psychological dimensions in a given text by counting the numbers of specific words [ 43 ] . it has been widely applied in researches involving social media and mental health. for the complete list of linguistic and psychological dimensions liwc measures, see [ 43(pp3-4) ]. we segmented the data log for each participant by mid-march as two blobs of texts and analyzed the words using liwc: for google search, we input the raw query text; for youtube, we input the video title. we considered the 'personal concerns', 'negative emotion', 'health/illness', and 'social words' liwc dimensions. liwc categorized words associated to work, leisure, home, money, and religion as 'personal concerns'. in the 'negative emotion' dimension, liwc included words related to anxiety, anger, and sadness. whereas, in the 'social words' dimension, liwc included family, all rights reserved. no reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted august 25, 2020. . friends, and gender references. the liwc output the count of words falling in each dimension among the whole text. we quantified the shift in behavior by calculating the percentage change of words in each dimension after the outbreak. we labeled each google search query with a category using the google nlp api [ 44 ]. we utilized the official youtube api to retrieve the information of videos watched by the participants, including the title, duration, number of likes and dislikes, and default youtube category tags. for a comprehensive list of google nlp category labels and default youtube category tags, please refer to [ 45, 46 ]. there were several categories overlapping with the liwc dimensions, such as 'health' and 'finance', and we regarded the liwc dimensions as a more well-studied standard. instead, we focused on the number of activities belonging to the 'adult' and 'news' categories, which were not presented in the liwc. we calculated the relative changes of activities in these two categories as the behavior shifts for each participant, same as equation 1 . there were in total 5 scalar continuous dependent variables measuring various aspects of the changes in online behavior for each participant, as defined above. these variables were extracted from two segments of the online data logs, namely the data before and after the pandemic outbreak. for the inactivity periods, the measurement was the kl-divergence between inactivity distributions. for the rest 4 behavioral features, the measurements were all in percentage changes. for both rounds of the data collection, anxiety levels were assessed using the gad-7 survey, and depression levels were assessed using the phq-9 survey. with two rounds of surveys reported before and after the outbreak, the change in mental health conditions of each participant were obtained. according to [ 47, 48 ] , an increase greater than or equals to 5 in the gad-7 score may be clinically alarming. therefore, individuals with an increase â³ 5 in gad-7 scores were labeled as the anx group; the rest were labeled as the non-anx group. similarly, as stated in [ 49 ] , an increase greater than or equals to 5 in the phq-9 score may indicate needs for medical interventions. hence, individuals with an increase â³ 5 in phq-9 scores were labeled as the dep group; the rest were labeled as the non-dep group. besides the online data and mental health surveys, we also collected basic demographic information such as school year, gender, and nationality. before any analysis on mental health conditions, in order to eliminate the possibility of annual confounding factors interfering the shifts in online behaviors, two-tailed paired independent t-tests were performed. we inspected that, in terms of the five quantitative features, whether the online behavior changes happened every year, such as due to seasonal factors, or only during covid-19 for the whole study population. as mentioned above, we collected the entire google history log back to the registration date of the google accounts of all participants. thus, we computed the online behaviors changes in both 2020 and 2019 for all participants, spanning 2.5 months before and after the mid-march of each year. the behavior changes were dependent between 2020 and 2019 for the same participant. viewing the cohort as a whole and measured twice, two-tailed paired independent t-tests were performed on all 5 behavior features. for the main experiment, chi-square tests were first performed to investigate the differences in demographics: school year, gender, and nationality. after that, analyses of covariance were conducted to explore the discrepancy between the dep and non-dep groups with each of the 5 online behavior features while controlling significant demographic covariates. the same was performed between the anx and non-anx groups. notice that, in this observational study, the independent variable was the binary group, i.e., whether or not the individual had a significant increase in the gad-7 (or phq-9) score. the dependent variables were the 5 behavior changes extracted from the longitudinal individual online data. experiments were carried out in a one-on-one fashion: anxiety or depression condition was the single independent variable, and one of the 5 online behavior changes was the single dependent variable each time. since multiple hypotheses were tested and some dependent variables might be moderately correlated, a holm's sequential bonferroni procedure were performed with an original significance level a=0.05 to deal with the family-wise error rates. we recruited 49 (n=49) participants in total, and all of them participated in both rounds of the study (response rate=100% of them reported an increase in the phq-9 score â³ 5 (the dep group); 45% (n=22) of them reported an increase in the gad-7 score â³ 5 (the anx group). 37% (n=18) of the participants belonged to the anx and dep group simultaneously. of the 49 participants, 61% (n=30) of the them were female; 35% (n=17) of the them were male; the rest 4% (n=2) reported non-binary genders. first and secondyear students occupied 63% (n=31) of the whole cohort, and the rest were third and fourth-year students (n=24). 80% (n=39) of the participants were u.s. citizens, and the rest (n=10) were international students. a complete breakdown of demographics and group separations are given in table 1 . the distributions of female participants were not well-stratified. 77% (n=17/22) of the anx group were female while 48% (n=13/27) of the non-anx group were female; 85% (n=17/20) of the dep group were female while 45% (n=13/29) of the non-dep group were female (table 1) . this observation among female students is consistent with the statistics reported in [ 4 ] . chi-square tests showed that being female had a significant difference between the anx and non-anx group (p=.07, ï� 7 ' =3.2); it also had a significant difference between the dep and non-dep group (p=.01, 7 ' =6.4). meanwhile, being an u.s. citizen did not show a significant difference in deteriorating anxiety (p=.99, 7 ' <0.1) nor depression (p=.76, 7 ' =0.1); being lower-class student (first or second-year) did not show a significant difference in deteriorating anxiety (p=.73, 7 ' =0.1) nor depression (p=.93, 7 ' <0.1). thus, the gender factor was controlled for the rest of the study. the two-tailed paired independent t-tests mentioned at the beginning of statistical analysis was designed to rule out seasonal factors in online behavior changes but focus on covid-19 before any of the main experiments, and they reported p<.001 for all 5 quantitative features. hence, the presence of annual or seasonal factors accountable for online behavior changes was neglectable, and it was safe to carry out the following main experiment. for each group (anx, non-anx, dep, and non-dep), the average percentage changes in late night activities, short event intervals, liwc attributes, and google search and youtube categories were all positive increases. analyses of covariance were performed to investigate the online behavior differences between the dep and the non-dep groups, ruling out the gender factor. we dummy-coded the categorical gender factor as a continuous covariate. for late night activities, the dep group (mean=9.70%, 95% ci 8.72%-10.68%) had a higher relative increase than the non-dep group (mean=7.54%, 95% ci 6.59%-8.49%), and a significant difference was found (p=. 005 for the liwc attributes, the dep group (mean=9.52%, 95% ci 8.36%-10.78%) had a higher relative increase in 'personal concern' keywords than the non-dep group (mean=7.24%, 95% ci 6.50%-7.98%), and a significant difference was found table 2 summarizes these findings in detail. figure 3 shows the distributions of the percentage increases in online behavior features except the inactivity divergence in the two groups. all rights reserved. no reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. similar trends were found between the anx and non-anx groups, partially due to the overlapping with the dep and non-dep populations. for late night activities all rights reserved. no reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted august 25, 2020. . https://doi.org/10.1101/2020.08.22.20178640 doi: medrxiv preprint (p=.001, !"#$%"& ' =0.231, f1,46=13.85), the anx group (mean=9.82%, 95% ci 9.04%-10.61%) had a higher percentage increase than the non-anx group (mean=7.28%, 95% ci 6.27%-8.29%). for inactivity periods (p=.01, !"#$%"& ' =0.135, f1,46=7.19), the anx group (mean=0.96, 95% ci 0.76-1.16) had a lower divergence, i.e., fewer alterations in the pattern of inactive periods in a 24-hour period, than the non-anx group (mean=1.27, 95% ci 1.13-1.42). the anx group (mean=18.09%, 95% ci 16.22%-19.95%) had more increase in short event intervals than the non-anx group (mean=14.64%, 95% ci13.21%-16.06%), and a significant difference was found (p=.007, !"#$%"& ' =0.149, f1,46=8.05) . for the liwc attributes, the anx group (mean=9.15%, 95% ci 7.88%-10.41%) had a higher relative increase in 'personal concern' keywords than the non-anx group (mean=7.37%, 95% ci 6.61%-8.14%), and this difference was statistically significant (p=. =0.003, f1,46=0.14) content showed any significant group difference. for more details, see table 3 . figure 4 shows the distributions of the percentage increases in online behavior features except the inactivity divergence in the two groups. (which was not certified by peer review) is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted august 25, 2020. . in this study, we collected longitudinal individual-level google search and youtube data from college students, and we measured their anxiety (gad-7) and depression (phq-9) levels before and after the outbreak of covid-19. we then developed explainable features from the online data logs and quantified the online behavior shifts of the participants during the pandemic. we also calculated the change in mental health conditions for all participants. our experiment examined the differences between groups with and without deteriorating mental health profiles in term of these online behavior features. to the best of our knowledge, we are the first to conduct observational studies on how mental health problems and google search and youtube usages of college students are related during covid-19. our results showed significant differences between groups of college students with and without worsened mental health profiles in terms of online behavior changes during the pandemic. the features we developed based on online activities were all explainable and preserved certain levels of interpretability. for example, the short event intervals and inactivity periods measured the consecutive usages and time away from google search and youtube, which were inspired by previous studies on excessive youtube usages [ 23 ] , internet addictions [ 50 ] , and positive associations with social anxiety among college students [ 24 ] . our results indicated that individuals with meaningful increasing anxiety or depressive disorders during the pandemic tended to have long usage sessions (multiple consecutive activities with short time intervals) when engaging with google search and youtube. moreover, anx and dep individuals tended to maintain their regular time-awayfrom-internet patterns regardless of the lockdown as the kl-divergence was low. one possible reason could be that depressed people tend to spend more time at home as regular lifestyles [ 37, 51 ] , and thereby, after the lockdown, the living environment did not alter much. we further found that the majority of the inactivity periods longer than 7 hours had midpoints around 5 to 6 a.m. for all individuals, which were most likely to be the sleeping period. we noticed well-establish previous researches stated that depressed individuals would have more disrupted sleeping patterns and less circadian lifestyles [ 37, 52, 53 ], but they are not validated for special periods such as covid-19. we instead focused on comparing the distributions of time away from google before and after the outbreak of covid-19, and we had an emphasis on the behavior changes of groups with and without worsened mental disorders. this implies that the resting inactive periods could be 'equally disrupted and irregular' before and after the outbreak for the dep group while the non-dep group underwent a major shift. besides, the increase in late night activities corresponded with previous studies in sleep depravations and subsequent positive correlations with mental health deteriorations [ 54, 55 ] . our results demonstrated that individuals with significant worsened anxiety or depressive symptoms during the pandemic were indeed likely to stay up late and engage more online. the above three features captured the temporal aspects of user online behaviors, and they have shown statistically significant differences between groups. additionally, our analysis found that there was significant difference in the amount of adult and porn consumption between individuals with and without worsening depression, which is adhere to previous findings that people suffering from depression and loneliness are likely to consume more pornographies [ 56, 57 ]. for the liwc features, 'personal concern' and 'negative emotion' key words appeared more frequently among students in the anx group, and previous research showed that negative youtube videos tended to receive more attention from vulnerable individuals [ 58 ] . for the dep group, 'social words' became less prevalent than the non-dep group. this was consistent with studies on patterns of social withdrawal and depression [ 37, 59, 60 ] , and social interactions and isolations have been recognized by [ 61 ] as one of the priorities in mental illness preventions, especially during covid-19 [ 27 ] . these attributes captured the semantic aspect of user online behaviors. the prevalence of personal affair, social activity, and negative key words as well as porn consumptions have shown statistically significant differences between groups. many researchers have reported that there has been a significant boost in health and news related topics, at population level, in various online platforms during covid-19. this is partly due to additional measures taken by individuals, various stakeholders, and agencies with regards to preventive measures [ 8, 32, 33 ], daily statistics [ 7, 9, 10 ], and healthcare (mis)information [ 31, 33, 34 ], however, unlike many, our investigation was carried out considering individual-level google search and youtube engagement logs, and our analysis did not reveal any significant spikes in 'news' and 'health/illness' category between the groups of individuals with deteriorating anxiety and depression during the pandemic. one possible explanation for such observation can be due to the target population (college students) of our study who may prefer to follow news from other popular platforms such as social media. finally, covid-19 has shaken the foundation of human society and forced us to alter daily lifestyles. the world was not ready for such a viral outbreak. since there is no cure for covid-19, it, or an even more deadly viral disease, may resurface at different capacities in the near future. society may be forced to rely on technologies even more and employ remote learning, working, and socializing for a longer period of time. it is important that we learn from our experience of living though the initial covid-19 outbreak and take necessary measures to uncover the changes in online behaviors, investigating how that can be leveraged to understand and monitor various mental health conditions of individuals in a least invasive manner. first, while most of the online behavioral features we developed showed significant differences between groups of students with and without deteriorating anxiety and depressive disorders during covid-19, our study cohort only represented a small portion of the whole population suffering from mental health difficulties. therefore, further studies are required to investigate if the significant behavioral changes still hold among more general communities, not limiting to college students. nonetheless, we argue that the explainable features we constructed, such as late-night activities, continuous usages, inactivity, pornography, and certain key words, can remain behaviorally representative and be applied universally across experiments exploring the relationship between mental health and online activities during the pandemic. second, in this work, we studied in the relationship between user online behaviors and the fluctuations in mental health conditions during covid-19. any causal relationship between online behavior and mental disorders is beyond the scope of all rights reserved. no reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted august 25, 2020. . this work. as one can readily imagine, online behavioral changes could both contribute to or be caused by deteriorating anxiety or depressive disorders. moreover, though we included preliminary demographic information as covariates, there remains the possibility of other confounding factors. in fact, both the shifts in online behaviors and deteriorating mental health profiles may be due to common factors such as living conditions, financial difficulties, and other health problems during the pandemic. nor there was any causal direction implied between covid-19 and online behavior changes, which was introduced in the first paragraph of statistical analysis as a precaution before the main experiments. albeit a pilot study, our results indicated that it is possible to build an anxiety and depression surveillance system based on passively collected private google data histories during covid-19. such non-invasive system shall be subject to rigorous data security and anonymity checks. necessary measures need to be in place to ensure personal safety and privacy concerns when collecting sensitive and proprietary data such as google search logs and youtube histories. even in pilot studies, participants shall preserve full rights over their data: they may choose to opt out the study at any stage and remove any data shared in the system. moreover, anonymity and systematic bias elimination shall be enforced. as an automatic medical screening system based on pervasive data, it has been extensively studied that such frameworks are prone to implicit machine learning bias during data collection or training phases [62] [63] [64] . black-box methods should be avoided as they are known to be vulnerable to adversarial attacks and produce unexplainable distributional representations [ 65, 66 ] . anonymizing data and obscuring identity information should be the first step in data debiasing. in the end, to what extent should caregivers trust a clinical decision made by machines remains an open question. we believe that possible pervasive computing frameworks shall play the role of a smart assistant, at most, to the care providers. any final intervention or help delivery decision should be made by healthcare professionals who understand both the mental health problems and the limitations of automatic detection systems in clinical settings. dep: the group of participants reported an increase in the phq-9 score â³ 5 liwc: linguistic inquiry and word count lna: late night activity non-anx: the group of participants did not report an increase in the gad-7 score â³ 5 non-dep: the group of participants did not report an increase in the phq-9 score â³ 5 sei: short event interval the outbreak of covid-19 coronavirus and its impact on global mental health mental health and the covid-19 pandemic covid-19 and mental health: a review of the existing literature students under lockdown: comparisons of students' social networks and mental health before and during the covid-19 crisis in switzerland prevalence, severity, and comorbidity of twelve-month dsm-iv disorders in the national comorbidity survey replication (ncs-r) failure and delay in initial treatment contact after first onset of mental disorders in the national comorbidity survey replication tracking covid-19 in europe: infodemiology approach. jmir public health and surveillance use of google trends to investigate loss-ofsmell-related searches during the covid-19 outbreak covid-19-related web search behaviors and infodemic attitudes in italy: infodemiological study. jmir public health and surveillance binge watching behavior during covid 19 pandemic: a cross-sectional, cross-national online survey mental health problems and social media exposure during covid-19 outbreak social networking sites, depression, and anxiety: a systematic review association between social media use and depression among u.s. young adults. depression and anxiety discovering shifts to suicidal ideation from mental health content in social media social media as a measurement tool of depression in populations detecting anxiety through reddit online social networks in health care: a study of mental disorders on reddit let me tell you about your mental health!": contextualized classification of reddit posts to dsm-5 for web-based intervention the stress-buffering effect of self-disclosure on facebook: an examination of stressful life events, social support, and mental health among college students youtube in the classroom: helpful tips and student perceptions social media as a complementary learning tool for teaching and learning: the case of youtube compulsive youtube usage: a comparison of use motivation and personality effects the relations between youtube addiction, social anxiety and parasocial relationships with youtubers: a moderated-mediation model based on a cognitive-behavioral framework suicide risk and prevention during the covid-19 pandemic. the lancet psychiatry using internet-based psychological measurement to capture the deteriorating community mental health profile during covid-19: observational study multidisciplinary research priorities for the covid-19 pandemic: a call for action for mental health science. the lancet psychiatry managing mental health challenges faced by healthcare workers during covid-19 pandemic mental health care for medical staff in china during the covid-19 outbreak. the lancet psychiatry patients with mental health disorders in the covid-19 epidemic. the lancet psychiatry youtube as a source of information on covid-19: a pandemic of misinformation? the role of youtube and the entertainment industry in saving lives by educating and mobilizing the public to adopt behaviors for community mitigation of covid-19: successive sampling design study preventive behaviors conveyed on youtube to mitigate transmission of covid-19: cross-sectional study. jmir public health and surveillance misinformation on social media: experimental evidence for a scalable accuracy-nudge intervention sense and sensibility: characterizing social media users regarding the use of controversial terms for covid-19 the relationship between mobile phone location sensor data and depressive symptom severity mobile phone sensor correlates of depressive symptom severity in daily-life behavior: an exploratory study objective assessment of depressive symptoms with machine learning and wearable sensors data detecting low self-esteem in youths from web search data on information and sufficiency the development and psychometric properties of liwc2015 a brief measure for assessing generalized anxiety disorder: the gad-7 psychometric properties of the generalized anxiety disorder scale-7 (gad-7) in outpatients with anxiety and mood disorders enhancing the clinical utility of depression screening internet addiction in students: prevalence and risk factors association between lifestyle activity and depressed mood among home-dwelling older people: a community-based study in japan circadian rhythms and depression: human psychopathology and animal models circadian pattern of motor activity in major depressed patients undergoing antidepressant therapy: relationship between actigraphic measures and clinical course sleep in adolescence: physiology, cognition and mental health reciprocal relationships between daily sleep and mood: a systematic review of naturalistic prospective studies understanding associations between personal definitions of pornography, using pornography, and depression youtube as a source of information on immunization: a content analysis nonverbal social withdrawal in depression: evidence from manual and automatic analyses. image and vision computing childhood social withdrawal, interpersonal impairment, and young adult depression: a mediational model artificial intelligence, bias and clinical safety potential biases in machine learning algorithms using electronic health record data a systematic review of machine learning models for predicting outcomes of stroke with structured data exploiting the vulnerability of deep learning-based artificial intelligence models in medical imaging: adversarial attacks understanding adversarial attacks on deep learning based medical image analysis systems. pattern recognition this research was supported in part by grant w911nf-15-1-0542 and w911nf-19-1-0029 with the us defense advanced research projects agency (darpa) and the army research office (aro). key: cord-021088-9u3kn9ge authors: huberty, mark title: awaiting the second big data revolution: from digital noise to value creation date: 2015-02-18 journal: nan doi: 10.1007/s10842-014-0190-4 sha: doc_id: 21088 cord_uid: 9u3kn9ge “big data”—the collection of vast quantities of data about individual behavior via online, mobile, and other data-driven services—has been heralded as the agent of a third industrial revolution—one with raw materials measured in bits, rather than tons of steel or barrels of oil. yet the industrial revolution transformed not just how firms made things, but the fundamental approach to value creation in industrial economies. to date, big data has not achieved this distinction. instead, today’s successful big data business models largely use data to scale old modes of value creation, rather than invent new ones altogether. moreover, today’s big data cannot deliver the promised revolution. in this way, today’s big data landscape resembles the early phases of the first industrial revolution, rather than the culmination of the second a century later. realizing the second big data revolution will require fundamentally different kinds of data, different innovations, and different business models than those seen to date. that fact has profound consequences for the kinds of investments and innovations firms must seek, and the economic, political, and social consequences that those innovations portend. yet this bbig data^revolution has so far fallen short of its promise. precious few firms transmutate data into novel products. instead, most rely on data to operate, at unprecedented scale, business models with long pedigree in the media and retail sectors. big data, despite protests to the contrary, is thus an incremental change-and its revolution one of degree, not kind. the reasons for these shortcomings point to the challenges we face in realizing the promise of the big data revolution. today's advances in search, e-commerce, and social media relied on the creative application of marginal improvements in computational processing power and data storage. in contrast, tomorrow's hopes for transforming real-world outcomes in areas like health care, education, energy, and other complex phenomena pose scientific and engineering challenges of an entirely different scale. our present enthusiasm for big data stems from the confusion of data and knowledge. firms today can gather more data, at lower cost, about a wider variety of subjects, than ever before. big data's advocates claim that this data will become the raw material of a new industrial revolution. as with its 19th century predecessor, this revolution will alter how we govern, work, play, and live. but unlike the 19th century, we are told, the raw materials driving this revolution are so cheap and abundant that the horizon is bounded only by the supply of smart people capable of molding these materials into the next generation of innovations (manyika et al. 2011) . this utopia of data is badly flawed. those who promote it rely on a series of dubious assumptions about the origins and uses of data, none of which hold up to serious scrutiny. in aggregate, these assumptions all fail to address whether the data we have actually provides the raw materials needed for a data-driven industrial revolution we need. taken together, these failures point out the limits of a revolution built on the raw materials that today seem so abundant. four of these assumptions merit special attention: first, n = all, or the claim that our data allow a clear and unbiased study of humanity; second, that today = tomorrow, or the claim that understanding online behavior today implies that we will still understand it tomorrow; third, offline = online, the claim that understanding online behavior offers a window into economic and social phenomena in the physical world; and fourth, that complex patterns of social behavior, once understood, will remain stable enough to become the basis of new data-driven, predictive products and services in sectors well beyond social and media markets. each of these has its issues. taken together, those issues limit the future of a revolution that relies, as today's does, on the bdigital exhaust^of social networks, e-commerce, and other online services. the true revolution must lie elsewhere. gathering data via traditional methods has always been difficult. small samples were unreliable; large samples were expensive; samples might not be representative, despite researchers' best efforts; tracking the same sample over many years required organizations and budgets that few organizations outside governments could justify. none of this, moreover, was very scalable: researchers needed a new sample for every question, or had to divine in advance a battery of questions and hope that this proved adequate. no wonder social research proceeded so slowly. mayer-schönberger and cukier (2013) argue that big data will eliminate these problems. instead of having to rely on samples, online data, they claim, allows us to measure the universe of online behavior, where n (the number of people in the sample) is basically all (the entire population of people we care about). hence we no longer need worry, they claim, about the problems that have plagued researchers in the past. when n = all, large samples are cheap and representative, new data on individuals arrives constantly, monitoring data over time poses no added difficulty, and cheap storage permits us to ask new questions of the same data again and again. with this new database of what people are saying or buying, where they go and when, how their social networks change and evolve, and myriad other factors, the prior restrictions borne of the cost and complexity of sampling will melt away. but n ≠ all. most of the data that dazzles those infatuated by bbig data^-mayer-schönberger and cukier included-comes from what mckinsey & company termed bdigital exhaust^ (manyika et al. 2011) : the web server logs, e-commerce purchasing histories, social media relations, and other data thrown off by systems in the course of serving web pages, online shopping, or person-to-person communication. the n covered by that data concerns only those who use these services-not society at large. in practice, this distinction turns out to matter quite a lot. the demographics of any given online service usually differ dramatically from the population at large, whether we measure by age, gender, race, education, and myriad other factors. hence the uses of that data are limited. it's very relevant for understanding web search behavior, purchasing, or how people behave on social media. but the n here is skewed in ways both known and unknown-perhaps younger than average, or more tech-savvy, or wealthier than the general population. the fact that we have enormous quantities of data about these people may not prove very useful to understanding society writ large. but let's say that we truly believe this assumption-that everyone is (or soon will be) online. surely the proliferation of smart phones and other devices is bringing that world closer, at least in the developed world. this brings up the second assumption-that we know where to go find all these people. several years ago, myspace was the leading social media website, a treasure trove of new data on social relations. today, it's the punchline to a joke. the rate of change in online commerce, social media, search, and other services undermines any claim that we can actually know that our n = all sample that works today will work tomorrow. instead, we only know about new developments-and the data and populations they cover-well after they have already become big. hence our n = all sample is persistently biased in favor of the old. moreover, we have no way of systematically checking how biased the sample is, without resorting to traditional survey methods and polling-the very methods that big data is supposed to render obsolete. but let's again assume that problem away. let's assume that we have all the data, about all the people, for all the online behavior, gathered from the digital exhaust of all the relevant products and services out there. perhaps, in this context, we can make progress understanding human behavior online. but that is not the revolution that big data has promised. most of the bbig data^hype has ambitions beyond improving web search, online shopping, socializing, or other online activity. instead, big data should help cure disease, detect epidemics, monitor physical infrastructure, and aid first responders in emergencies. to satisfy these goals, we need a new assumption: that what people do online mirrors what they do offline. otherwise, all the digital exhaust in the world won't describe the actual problems we care about. there's little reason to think that offline life faithfully mirrors online behavior. research has consistently shown that individuals' online identities vary widely from their offline selves. in some cases, that means people are more cautious about revealing their true selves. danah boyd's work (boyd and marwick 2011) has shown that teenagers cultivate online identities very different from their offline selves-whether for creative, privacy, or other reasons. in others, it may mean that people are more vitriolic, or take more extreme positions. online political discussions-another favorite subject of big data enthusiasts-suffer from levels of vitriol and partisanship far beyond anything seen offline (conover et al. 2011) . of course, online and offline identity aren't entirely separate. that would invite suggestions of schizophrenia among internet users. but the problem remains-we don't know what part of a person is faithfully represented online, and what part is not. furthermore, even where online behavior may echo offline preferences or beliefs, that echo is often very weak. in statistical terms, our ability to distinguish bsignificant^from binsignificant^results improves with the sample size-but statistical significance is not actual significance. knowing, say, that a history of purchasing some basket of products is associated with an increased risk of being a criminal may be helpful. but if that association is weak-say a one-hundredth of a percent increase-it's practical import is effectively zero. big data may permit us to find these associations, but it does not promise that they will be useful. ok, but you say, surely we can determine how these distortions work, and incorporate them into our models? after all, doesn't statistics have a long history of trying to gain insight from messy, biased, or otherwise incomplete data? perhaps we could build such a map, one that allows us to connect the observed behaviors of a skewed and selective online population to offline developments writ large. this suffices only if we care primarily about describing the past. but much of the promise of big data comes from predicting the future-where and when people will get sick in an epidemic, which bridges might need the most attention next month, whether today's disgruntled high school student will become tomorrow's mass shooter. satisfying these predictive goals requires yet another assumption. it is not enough to have all the data, about all the people, and a map that connects that data to real-world behaviors and outcomes. we also have to assume that the map we have today will still describe the world we want to predict tomorrow. two obvious and unknowable sources of change stand in our way. first, people change. online behavior is a culmination of culture, language, social norms and other factors that shape both people and how they express their identity. these factors are in constant flux. the controversies and issues of yesterday are not those of tomorrow; the language we used to discuss anger, love, hatred, or envy change. the pathologies that afflict humanity may endure, but the ways we express them do not. second, technological systems change. the data we observe in the bdigital exhaust^of the internet is created by individuals acting in the context of systems with rules of their own. those rules are set, intentionally or not, by the designers and programmers that decide what we can and cannot do with them. and those rules are in constant flux. what we can and cannot buy, who we can and cannot contact on facebook, what photos we can or cannot see on flickr vary, often unpredictably. facebook alone is rumored to run up to a thousand different variants on its site at one time. hence even if culture never changed, our map from online to offline behavior would still decay as the rules of online systems continued to evolve. an anonymous reviewer pointed out, correctly, that social researchers have always faced this problem. this is certainly true but many of the features of social systems-political and cultural institutions, demography, and other factors-change on a much longer timeframe than today's data-driven internet services. for instance, us congressional elections operate very differently now compared with a century ago; but change little between any two elections. contrast that with the pace of change for major social media services, for which 2 years may be a lifetime. a recent controversy illustrates this problem to a t. facebook recently published a study (kramer et al. 2014 ) in which they selectively manipulated the news feeds of a randomized sample of users, to determine whether they could manipulate users' emotional states. the revelation of this study prompted fury on the part of users, who found this sort of manipulation unpalatable. whether they should, of course, given that facebook routinely runs experiments on its site to determine how best to satisfy (i.e., make happier) its users, is an interesting question. but the broader point remains-someone watching the emotional state of facebook users might have concluded that overall happiness was on the rise, perhaps consequence of the improving american economy. but in fact this increase was entirely spurious, driven by facebook's successful experiment at manipulating its users. compounding this problem, we cannot know, in advance, which of the social and technological changes we do know about will matter to our map. that only becomes apparent in the aftermath, as real-world outcomes diverge from predictions cast using the exhaust of online systems. lest this come off as statistical nihilism, consider the differences in two papers that both purport to use big data to project the outcome of us elections. digrazia et al. (2013) claim that merely counting the tweets that reference a congressional candidate-with no adjustments for demography, or spam, or even name confusion-can forecast whether that candidate will win his or her election. this is a purely bdigital exhaust^approach. they speculate-but cannot know-whether this approach works because (to paraphrase their words) bone tweet equals one vote^, or ball attention on twitter is better^. moreover, it turns out that the predictive performance of this simple model provides no utility. as huberty (2013) shows, their estimates perform no better than an approach that simply guesses that the incumbent party would win-a simple and powerful predictor of success in american elections. big data provided little value. contrast this with wang et al. (2014) . they use the xbox gaming platform as a polling instrument, which they hope might help compensate for the rising non-response rates that have plagued traditional telephone polls. as with twitter, n ≠ all: the xbox user community is younger, more male, less politically involved. but the paper nevertheless succeeds in generating accurate estimates of general electoral sentiment. the key difference lies in their use of demographic data to re-weight respondents' electoral sentiments to look like the electorate at large. the xbox data were no less skewed than twitter data; but the process of data collection provided the means to compensate. the black box of twitter's digital exhaust, lacking this data, did not. the difference? digrazia et al. (2013) sought to reuse data created for one purpose in order to do something entirely different; wang et al. (2014) set out to gather data explicitly tailored to their purpose alone. 2.5 the implausibility of big data 1.0 taken together, the assumptions that we have to make to fulfill the promise of today's big data hype appear wildly implausible. to recap, we must assume that: 1. everyone we care about is online; 2. we know where to find them today, and tomorrow; 3. they represent themselves online consistent with how they behave offline, and; 4. they will continue to represent themselves online-in behavior, language, and other factors-in the same way, for long periods of time. nothing in the history of the internet suggests that even one of these statements holds true. everyone was not online in the past; and likely will not be online in the future. the constant, often wrenching changes in the speed, diversity, and capacity of online services means those who are online move around constantly. they do not, as we've seen, behave in ways necessarily consistent with their offline selves. and the choices they make about how to behave online evolve in unpredictable ways, shaped by a complex and usually opaque amalgam of social norms and algorithmic influences. but if each of these statements fall down, then how have companies like amazon, facebook, or google built such successful business models? the answer lies in two parts. first, most of what these companies do is self-referential: they use data about how people search, shop, or socialize online to improve and expand services targeted at searching, shopping, or socializing. google, by definition, has an n = all sample of google users' online search behavior. amazon knows the shopping behaviors of amazon users. of course, these populations are subject to change their behaviors, their self-representation, or their expectations at any point. but at least google or amazon can plausibly claim to have a valid sample of the primary populations they care about. second, the consequences of failure are, on the margins, very low. google relies heavily on predictive models of user behavior to sell the advertising that accounts for most of its revenue. but the consequences of errors in that model are low-google suffers little from serving the wrong ad on the margins. of course, persistent and critical errors of understanding will undermine products and lead to lost customers. but there's usually plenty of time to correct course before that happens. so long as google does better than its competitors at targeting advertising, it will continue to win the competitive fight for advertising dollars. but if we move even a little beyond these low-risk, self-referential systems, the usefulness of the data that underpin them quickly erodes. google flu provides a valuable lesson in this regard. in 2008, google announced a new collaboration with the centers for disease control (cdc) to track and report rates of influenza infection. historically, the cdc had monitored us flu infection patterns through a network of doctors that tracked and reported binfluenza-like illness^in their clinics and hospitals. but doctors' reports took up to 2 weeks to reach the cdc-a long time in a world confronting sars or avian flu. developing countries with weaker public health capabilities faced even greater challenges. google hypothesized that, when individuals or their family members got the flu, they went looking on the internet-via google, of course-for medical advice. in a highly cited paper, ginsberg et al. (2008) showed that they could predict region-specific influenza infection rates in the united states using google search frequency data. here was the true promise of big data-that we capitalize on virtual data to better understand, and react to, the physical world around us. the subsequent history of google flu illustrates the shortcomings of the first big data revolution. while google flu has performed well in many seasons, it has failed twice, both times in the kind of abnormal flu season during which accurate data are most valuable. the patterns of and reasons for failure speak to the limits of prediction. in 2009, google flu underpredicted flu rates during the h1n1 pandemic. post-hoc analysis suggested that the different viral characteristics of h1n1 compared with garden-variety strains of influenza likely meant that individuals didn't know they had a flu strain, and thus didn't go looking for flu-related information (cook et al. 2011) . conversely, in 2012, google flu over-predicted influenza infections. google has yet to discuss why, but speculation has centered on the intensive media coverage of an early-onset flu season, which may have sparked interest in the flu among healthy individuals (butler 2013 ). the problems experienced by google flu provide a particularly acute warning of the risks inherent in trying to predict what will happen in the real world based on the exhaust of the digital one. google flu relied on a map-a mathematical relationship between online behavior and real-world infection. google built that map on historic patterns of flu infection and search behavior. it assumed that such patterns would continue to hold in the future. but there was nothing fundamental about those patterns. either a change in the physical world (a new virus) or the virtual one (media coverage) were enough to render the map inaccurate. the cdc's old reporting networks out-performed big data when it mattered most. despite ostensibly free raw materials, mass-manufacturing insight from digital exhaust has thus proven far more difficult than big data's advocates would let on. it's thus unsurprising that this revolution has had similarly underwhelming effects on business models. amazon, facebook, and google are enormously successful businesses, underpinned by technologies operating at unprecedented scale. but they still rely on centuries-old business models for most of their revenue. google and amazon differ in degree, but not kind, from a newspaper or a large department store when it comes to making money. this is a weak showing from a revolution that was supposed to change the 21st century in the way that steam, steel, or rail changed the 19th. big data has so far made it easier to sell things, target ads, or stalk long-lost friends or lovers. but it hasn't yet fundamentally reworked patterns of economic life, generated entirely new occupations, or radically altered relationships with the physical world. instead, it remains oddly self-referential: we generate massive amounts of data in the process of online buying, viewing, or socializing; but find that data truly useful only for improving online sales and search. understanding how we might get from here to there requires a better understanding of how and why data-big or small-might create value in a world of better algorithms and cheap compute capacity. close examination shows that firms have largely used big data to improve on existing business models, rather than adopt new ones; and that those improvements have relied on data to describe and predict activity in worlds largely of their own making. where firms have ventured beyond these self-constructed virtual worlds, the data have proven far less useful, and products built atop data far more prone to failure. the google flu example suggests the limits to big data as a source of mass-manufactured insight about the real world. but google itself, and its fellow big-data success stories, also illustrate the shortcomings of big data as a source of fundamentally new forms of value creation. most headline big data business models have used their enhanced capacity to describe, predict, or infer in order to implement-albeit at impressive scale and complexity-centuries-old business models. those models create value not from the direct exchange between consumer and producer, but via a web of transactions several orders removed from the creation of the data itself. categorizing today's big data business models based on just how far they separate data generation from value creation quickly illustrates how isolated the monetary value of firms' data is from their primary customers. having promised a first-order world, big data has delivered a third-order reality. realizing the promise of the big data revolution will require a different approach. the same problems that greeted flu prediction have plagued other attempts to build big data applications that forecast the real world. engineering solutions to these problems that draw on the potential of cheap computation and powerful algorithms will require not different methods, but different raw materials. the data those materials require must originate from a first-order approach to studying and understanding the worlds we want to improve. such approaches will require very different models of firm organization than those exploited by google and its competitors in the first big data revolution. most headline big data business models do not make much money directly from their customers. instead, they rely on third parties-mostly advertisers-to generate profits from data. the actual creation and processing of data is only useful insofar as it's of use to those third parties. in doing so, these models have merely implemented, at impressive scale and complexity, the very old business model used by the newspapers they have largely replaced. if we reach back into the dim past when newspapers were viable businesses (rather than hobbies of the civic-minded wealthy), we will remember that their business model had three major components: 1. gather, filter, and analyze news; 2. attract readers by providing that news at far below cost, and; 3. profit by selling access to those readers to advertisers. the market for access matured along with the newspapers that provided it. both newspapers and advertisers realized that people who read the business pages differed from those who read the front page, or the style section. front-page ads were more visible to readers than those buried on page a6. newspapers soon started pricing access to their readers accordingly. bankers paid one price to advertise in the business section, clothing designers another for the style pages. this segmentation of the ad market evolved as the ad buyers and sellers learned more about whose eyeballs were worth how much, when, and where. newspapers were thus third-order models. the news services they provided were valuable in their own right. but readers didn't pay for them. instead, news was a means of generating attention and data, which was only valuable when sold to third parties in the form of ad space. data didn't directly contribute to improving the headline product-news-except insofar as it generated revenue that could be plowed back into news gathering. the existence of a tabloid press of dubious quality but healthy revenues proved the weakness of the link between good journalism and profit. from a value creation perspective, google, yahoo, and other ad-driven big data businesses are nothing more than newspapers at scale. they too provide useful services (then news, now email or search) to users at rates far below cost. they too profit by selling access to those users to third-party advertisers. they too accumulate and use data to carve up the ad market. the scale of data they have available, of course, dwarfs that of their newsprint ancestors. this data, combined with cheap computation and powerful statistics, has enabled operational efficiency, scale, and effectiveness far beyond what newspapers could ever have managed. but the business model itself-the actual means by which these firms earn revenues-is identical. finally, that value model does not emerge, fully-formed, from the data itself. the data alone are no more valuable than the unrefined iron ore or crude oil of past industrial revolutions. rather, the data were mere inputs to a production process that depended on human insightthat what people looked for on the internet might be a good proxy for their consumer interests. big-box retail ranks as the other substantial success for big data. large retailers like amazon, wal-mart, or target have harvested very fine-grained data about customer preferences to make increasingly accurate predictions of what individual customers wish to buy, in what quantities and combinations, at what times of the year, at what price. these predictions are occasionally shocking in their accuracy-as with target's implicit identification of a pregnant teenager well before her father knew it himself, based solely on subtle changes in her purchasing habits. from this data, these retailers can, and have, built a detailed understanding of retail markets: what products are complements or substitutes for each other; exactly how much more people are willing to pay for brand names versus generics; how size, packaging, and placement in stores and on shelves matters to sales volumes. insights built on such data have prompted two significant changes in retail markets. first, they have made large retailers highly effective at optimizing supply chains, identifying retail trends in their infancy, and managing logistical difficulties to minimize the impact on sales and lost competitiveness. this has multiplied their effectiveness versus smaller retailers, who lack such capabilities and are correspondingly less able to compete on price. but it has also changed, fundamentally, the relationship of these retailers to their suppliers. big box retailers have increasingly become monopsony buyers of some goods-books for amazon, music for itunes. but they are also now monopoly sellers of information back to their suppliers. amazon, target and wal-mart have a much better understanding of their suppliers' customers than the customers themselves. they also understand these suppliers' competitors far better. hence their aggregation of information has given them substantial power over suppliers. this has had profound consequences for the suppliers. wal-mart famously squeezes suppliers on cost-either across the board, or by pitting suppliers against one another based on detailed information of their comparative cost efficiencies and customer demand. hence big data has shifted the power structure of the retail sector and its manufacturing supply chains. the scope and scale of the data owned by amazon or wal-mart about who purchases what, when, and in what combinations often means that they understand the market for a product far better than the manufacturer. big data, in this case, comes from big business-a firm that markets to the world also owns data about the world's wants, needs, and peculiarities. even as they are monopsony buyers of many goods (think e-books for amazon), they are correspondingly monopoly sellers of data. and that has made them into huge market powers on two dimensions, enabling them to squeeze suppliers to the absolute minimum price, packaging, size, and other product features that are most advantageous to them-and perhaps to their customers. but big data has not changed the fundamental means of value creation in the retail sector. whatever its distributional consequences, the basic retail transaction-of individuals buying goods from retail intermediaries, remains unchanged from earlier eras. the same economies of scale and opportunities for cross-marketing that made montgomery ward a retail powerhouse in the 19th century act on amazon and wal-mart in the 21st. big data may have exacerbated trends already present in the retail sector; but the basics of how that sector creates value for customers and generates profits for investors are by no means new. retailers have yet to build truly new products or services that rely on data itself-instead, that data is an input into a longstanding process of optimization of supply chain relations, marketing, and product placement in service of a very old value model: the final close of sale between a customer and the retailer. second-and third-order models find value in data several steps removed from the actual transaction that generates the data. however, as the google flu example illustrated, that data may have far less value when separated from its virtual context. thus while these businesses enjoy effectively free raw materials, the potential uses of those materials are in fact quite limited. digital exhaust from web browsing, shopping, or socializing has proven enormously useful in the self-referential task of improving future web browsing, shopping, and socializing. but that success has not translated success at tasks far removed from the virtual world that generated this exhaust. digital exhaust may be plentiful and convenient to collect, but it offers limited support for understanding or responding to real-world problems. first-order models, in contrast, escape the flu trap by building atop purpose-specific data, conceived and collected with the intent of solving specific problems. in doing so, they capitalize on the cheap storage, powerful algorithms, and inexpensive computing power that made the first wave of big data firms possible. but they do so in pursuit of a rather different class of problems. first order products remain in their infancy. but some nascent examples suggest what might be possible. ibm's watson famously used its natural language and pattern recognition abilities to win the jeopardy! game show. doing so constituted a major technical feat: the ability to understand unstructured, potentially obfuscated jeopardy! game show answers, and respond with properly-structured questions based on information gleaned from vast databases of unstructured information on history, popular culture, art, science, or almost any other domain. the question now is whether ibm can adapt this technology to other problems. its first attempts at improving medical diagnosis appear promising. by learning from disease and health data gathered from millions of patients, initial tests suggest that watson can improve the quality, accuracy, and efficacy of medical diagnosis and service to future patients (steadman 2013) . watson closes the data value loop: patient data is made valuable because it improves patient services, not because it helps with insurance underwriting or product manufacturing or logistics or some other third-party activity. premise corporation provides another example. premise has built a mobile-phone based data gathering network to measure macroeconomic aggregates like inflation and food scarcity. this network allows them to monitor economic change at a very detailed level, in regions of the world where official statistics are unavailable or unreliable. this sensor network is the foundation of the products and services that premise sells to financial services firms, development agencies, and other clients. as compared with the attenuated link between data and value in second-or third-order businesses, premise's business model links the design of the data generation process directly to the value of its final products. optimum energy (oe) provides a final example. oe monitors and aggregates data on building energy use-principally data centers-across building types, environments, and locations. that data enables it to build models for building energy use and efficiency optimization. those models, by learning building behaviors across many different kinds of inputs and buildings, can perform better than single-building models with limited scope. most importantly, oe creates value for clients by using this data to optimize energy efficiency and reduce energy costs. these first-order business models all rely on data specifically obtained for their products. this reliance on purpose-specific data contrasts with third-order models that rely on the bdigital exhaust^of conventional big data wisdom. to use the newspaper example, thirdorder models assume-but can't specifically verify-that those who read the style section are interested in purchasing new fashions. google's success stemmed from closing this information gap a bit-showing that people who viewed web pages on fashion were likely to click on fashion ads. but again, the data that supports this is data generated by processes unrelated to actual purchasing-activities like web surfing and search or email exchange. and so the gap remains. google appears to realize this, and has launched consumer surveys as an attempt to bridge that gap. in brief, it offers people the chance to skip ads in favor of providing brand feedback. we should remember the root of the claim about big data. that claim was perhaps best summarized by halevy et al. (2009) in what they termed bthe unreasonable effectiveness of data^-that, when seeking to improve the performance of predictive systems, more data appeared to yield better returns on effort than better algorithms. most appear to have taken that to mean that data-and particularly more data-are unreasonably effective everywhereand that, by extension, even noisy or skewed data could suffice to answer hard questions if we could simply get enough of it. but that misstates the authors' claims. they did not claim that more data was always better. rather, they argued that, for specific kinds of applications, history suggested that gathering more data paid better dividends than inventing better algorithms. where data are sparse or the phenomenon under measurement noisy, more data allow a more complete picture of what we are interested in. machine translation provides a very pertinent example: human speech and writing varies enormously within one language, let alone two. faced with the choice between better algorithms for understanding human language, and more data to quantify the variance in language, more data appears to work better. but for other applications, the bbigness^of data may not matter at all. if i want to know who will win an election, polling a thousand people might be enough. relying on the aggregated voices of a nation's twitter users, in contrast, will probably fail (gayo-avello et al. 2011; gayo-avello 2012; huberty 2013) . not only are we not, as section 2 discussed, in the n = all world that infatuated mayer-schönberger and cukier (2013); but for most problems we likely don't care to be. having the right data-and consequently identifying the right question to ask beforehand-is far more important than having a lot of data of limited relevance to the answers we seek. big data therefore falls short of the proclamation that it represents the biggest change in technological and economic possibility since the industrial revolution. that revolution, in the span of a century or so, fundamentally transformed almost every facet of human life. someone born in 1860, who lived to be 70 years old, grew up in a world of horses for travel, candles for light, salting and canning for food preservation, and telegraphs for communication. the world of their passing had cars and airplanes, electric light and refrigerators, and telephones, radio, and motion pictures. having ranked big data with the industrial revolution, we find ourselves wondering why our present progress seems so paltry in comparison. but much of what we associate with the industrial revolution-the advances in automobile transport, chemistry, communication, and medicine-came much later. the businesses that produced them were fundamentally different from the small collections of tinkerers and craftsmen that built the first power looms. instead, these firms invested in huge industrial research and development operations to discover and then commercialize new scientific discoveries. these changes were expensive, complicated, and slow-so slow that john stuart mill despaired, as late as 1871, of human progress. but in time, they produced a world inconceivable to even the industrial enthusiasts of the 1840s. in today's revolution, we have our looms, but we envision the possibility of a model t. today, we can see glimmers of that possibility in ibm's watson, google's self-driving car, or nest's thermostats that learn the climate preferences of a home's occupants. these and other technologies are deeply embedded in, and reliant on, data generated from and around realworld phenomena. none rely on bdigital exhaust^. they do not create value by parsing customer data or optimizing ad click-through rates (though presumably they could). they are not the product of a relatively few, straightforward (if ultimately quite useful) insights. instead, ibm, google, and nest have dedicated substantial resources to studying natural language processing, large-scale machine learning, knowledge extraction, and other problems. the resulting products represent an industrial synthesis of a series of complex innovations, linking machine intelligence, real-time sensing, and industrial design. these products are thus much closer to what big data's proponents have promised-but their methods are a world away from the easy hype about mass-manufactured insights from the free raw material of digital exhaust. we're stuck in the first industrial revolution. we have the power looms and the water mills, but wonder, given all the hype, at the absence of the model ts and telephones of our dreams. the answer is a hard one. the big gains from big data will require a transformation of organizational, technological, and economic operations on par with that of the second industrial revolution. then, as now, firms had to invest heavily in industrial research and development to build the foundations of entirely new forms of value creation. those foundations permitted entirely new business models, in contrast to the marginal changes of the first industrial revolution. and the raw materials of the first revolution proved only tangentially useful to the innovations of the second. these differences portend a revolution of greater consequence and complexity. firms will likely be larger. innovation will rely less on small entrepreneurs, who lack the funds and scale for systems-level innovation. where entrepreneurs do remain, they will play far more niche roles. as rao (2012) has argued, startups will increasingly become outsourced r&d, whose innovations are acquired to become features of existing products rather than standalone products themselves. the success of systems-level innovation will threaten a range of current jobs-white collar and service sector as well as blue collar and manufacturing-as expanding algorithmic capacity widens the scope of digitizeable tasks. but unlike past revolutions, that expanding capacity also begs the question of where this revolution will find new forms of employment insulated from these technological forces; and if it does not, how we manage the social instability that will surely follow. with luck, we will resist the temptation to use those same algorithmic tools for social control. but human history on that point is not encouraging. regardless, we should resist the temptation to assume that a world of ubiquitous data means a world of cheap, abundant, and relevant raw materials for a new epoch of economic prosperity. the most abundant of those materials today turn out to have limited uses outside the narrow products and services that generate them. overcoming that hurdle requires more than just smarter statisticians, better algorithms, or faster computation. instead, it will require new business models capable of nurturing both new sources of data and new technologies into truly new products and services. social privacy in networked publics: teens' attitudes, practices, and strategies when google got flu wrong assessing google flu trends performance in the united states during the 2009 influenza virus a (h1n1) pandemic more tweets, more votes: social media as a quantitative indicator of political behavior d (2012) i wanted to predict elections with twitter and all i got was this lousy paper: a balanced survey on election prediction using twitter data limits of electoral predictions using twitter detecting influenza epidemics using search engine query data the unreasonable effectiveness of data multi-cycle forecasting of congressional elections with social media experimental evidence of massive-scale emotional contagion through social networks big data: the next frontier for innovation, competition, and productivity. mckinsey global institute report mayer-schönberger v, cukier k (2013) big data: a revolution that will transform how we live, work, and think entrepreneurs are the new labor ibm's watson is better at diagnosing cancer than human doctors forecasting elections with non-representative polls acknowledgments this research is a part of the ongoing collaboration of brie, the berkeley roundtable on the international economy at the university of california at berkeley, and etla, the research institute of the finnish economy. this paper has benefited from extended discussions with cathryn carson, drew conway, chris diehl, stu feldman, david gutelius, jonathan murray, joseph reisinger, sean taylor, georg zachmann, and john zysman. all errors committed, and opinions expressed, remain solely my own.open access this article is distributed under the terms of the creative commons attribution license which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited. key: cord-298953-9aifql2f authors: day, brett h. title: the value of greenspace under pandemic lockdown date: 2020-08-04 journal: environ resour econ (dordr) doi: 10.1007/s10640-020-00489-y sha: doc_id: 298953 cord_uid: 9aifql2f the covid-19 outbreak resulted in unprecedented restrictions on citizen’s freedom of movement as governments moved to institute lockdowns designed to reduce the spread of the virus. while most out-of-home leisure activities were prohibited, in england the lockdown rules allowed for restricted use of outdoor greenspace for the purposes of exercise and recreation. in this paper, we use data recorded by google from location-enabled mobile devices coupled with a detailed recreation demand model to explore the welfare impacts of those constraints on leisure activities. our analyses reveals evidence of large-scale substitution of leisure time towards recreation in available greenspaces. indeed, despite the restrictions the economic value of greenspace to the citizens of england fell by only £150 million over lockdown. examining the outcomes of counterfactual policies we find that the imposition of stricter lockdown rules would have reduced welfare from greenspace by £1.14 billion. in contrast, more relaxed lockdown rules would have delivered an aggregate increase in the economic value of greenspace equal to £1.47 billion. as the covid-19 pandemic swept across the planet, national governments instituted various rules designed to reduce human contact and slow rates of infection. the severity of these lockdown rules differed from nation to nation, largely mirroring the severity of the virus outbreak. this paper focuses on england, whose own lockdown experience began on 23rd march, 2020. the lockdown in england placed unprecedented restrictions on citizen's freedom of movement. as well as not being able to go to their places of work, citizens were deprived of access to most shops, food and drink outlets, entertainment establishments and leisure facilities. one of the few privileges that remained was the opportunity to spend time outdoors walking and exercising, activity often undertaken in greenspace. this paper presents an empirical exploration of the levels of engagement with greenspace over the lockdown in england. it focuses on the question of how greatly the lockdown rules impacted on the value flows realised by english citizens from their greenspace and explores how those impacts might have differed had stricter or more relaxed restrictions been imposed. a priori, it is not self-evident whether the value derived from greenspace as a focus for outdoor recreation was diminished or amplified by the rules of lockdown and the conditions of the covid-19 outbreak. on the one hand, citizens may have reduced their use of greenspaces in an effort to minimise their risks of exposure to the virus. likewise, lockdown rules prevented citizens from visiting all but highly local greenspaces. limiting citizens' options to a small set of potentially less-desirable destinations will again have acted to dampen demand. on the other hand, under lockdown, citizens were unable to participate in nearly all other forms of out-of-home leisure activity, demand for greenspace may have increased as citizens substituted away from those unavailable alternative uses of their leisure time. in addition, under lockdown many citizens were unable to work. releasing the usual-leisure time constraints on those individuals will also have acted to increase demand for outdoor recreation. as the lockdown unfolded, localised evidence of changing behaviour arose. newspaper reports described normally busy beaches as all but deserted (betts 2020; crane 2020; ikonen 2020) . in contrast, incidents of overcrowding in city greenspaces resulted in temporary closures of several large urban parks (including london's brockwell park and victoria park as well as middlesbrough's stewart park). in this paper we make use of data collected by google from location-enabled mobile devices which provides systematic evidence on the rates of visitation to greenspace across the regions of england over the course of the lockdown (google 2020). as described in sect. 3, this google mobility data reveals that demand for greenspace changed over the course of the lockdown in ways which mirror the evolving rules on outdoor activity. the second key resource used in this paper is the outdoor recreation valuation (orval) model (day and smith 2017) , which we use not only to predict demand for visits to greenspace under the restrictive rules of the lockdown but also to estimate the changes in economic value experienced by residents of england as a consequence of those rules. developed in partnership with the uk government, 1 orval is underpinned by an econometric model estimated in the random utility framework. as such, orval follows in a tradition stretching back at least as far as kocur et al. (1979) and feenberg and mills (1980) , what distinguishes orval from other such models is that it is, as far as we are aware, the first to consider the entire range of publically-accessible greenspace sites including parks, gardens, playing fields, church yards, cemeteries, allotments, nature reserves, woodlands, wetlands, river and lakeside walks, beaches and the network of coastal and countryside paths. we briefly review the orval model in sect. 4. of course, orval was estimated on data in which individuals were not concerned with risking exposure to a deadly virus, in which their pursuit of alternative leisure activities was unrestricted and where they faced the leisure-time constraints of normal working conditions. in this paper, we assume that differences between the orval predictions of recreation behaviour under the lockdown rules and those observed in the google mobility data are the net result of those, and possibly other, factors. as described in sect. 5, we undertake a novel statistical exercise in model calibration using techniques of latent class regression to estimate parameters for the orval model which capture the net effect of those factors on recreation behaviour. those estimates allow us to construct a times series of orval predictions for recreation activity under the rules of the lockdown that can be contrasted to a counterfactual in which covid-19 had not come to pass. we present the findings from that comparison in sect. 6. in brief, we find that while the lockdown imposed very significant restrictions on outdoor recreation activities, citizens engaged in substantial compensating substitution behaviour. the mitigating effect of that substitution behaviour meant that over the lockdown, citizens of england experienced only a 2.1% fall in the welfare they might otherwise have enjoyed from greenspace, an amount equating to a loss in aggregate economic value of £150 million. our calibration of the orval model allows us to explore other counterfactuals; namely, how engagement with the outdoors might have proceeded through the covid-19 outbreak under stricter or under more relaxed lockdown rules. not surprisingly, we find that in the strict-rules counterfactual welfare from greenspace is £1.14 billion lower than under the actual lockdown rules. in contrast, applying less strict lockdown rules on outdoor recreation allows for even greater use of the outdoors and delivers an aggregate welfare benefit of £1.47 billion. this paper's contribution is primarily empirical. it attempts to quantify the impact of the covid-19 pandemic and its associated lockdown on one aspect of a nation's everyday life; outdoor recreation in greenspace. not surprisingly, given the recency of the events, little exists in the published literature with a similar intent. an unpublished manuscript by venter et al. (2020) examines changes in outdoor activity in oslo, norway during the virus outbreak. using data on the route choices of runners and cyclists, they find that spatial patterns of exercise activity changed over lockdown to favour greener and more remote locations. through a calibration exercise, venter et al. estimate that outdoor recreation activity in oslo increased by291%. in another yet to be published manuscript, rice and pan (2020) explore data made publically available by google on the use of greenspace during the covid-19 pandemic, data that we also exploit in our study (google 2020) . focusing on 111 counties in the western united states, they identify an average 2.5% increase in greenspace visitation and find that differences across counties are chiefly explained through differences in weather. 2 our study differs from these other contributions in a number of ways. the focus of our study is england, where lockdown rules on recreation were not dissimilar to those in the western us but significantly stricter than in oslo. 3 rather than routes used for exercise we explore visits to greenspace. and unlike both venter et al. (2020) and rice and pan (2020) , our focus is not primarily on how recreation patterns changed over space, but how they responded to changes in lockdown rules. perhaps the clearest point of separation is that we are the first to attempt to attribute an economic value to the changes in greenspace use that arose over the lockdown. the english lockdown began on march 23rd, 2020 with non-essential workers asked to work from home. shops and entertainment outlets were forced to close unless selling essential items and travel was only allowed if absolutely necessary. our particular interest concerns the rules on outdoor recreation for which specific guidelines were issued people were expected to use open spaces near to their homes and encouraged to limit themselves to one trip a day. driving to open spaces for the purposes of outdoor recreation was not allowed (hc deb 24th march 2020). requirement to abide by these measures was passed into law under the uk coronavirus act (2020) giving police the authority to issue fines of up to £960 to those that did not comply. after seven weeks of strict lockdown rules in the uk, outdoor recreation was amongst the first areas of daily life to experience a loosening of restrictions. in his televised speech to the british public on 10th may 2020, the british prime minister stated that, "we want to encourage people to take more and even unlimited amounts of outdoor exercise. you can sit in the sun in your local park, you can drive to other destinations, you can even play sports" (johnson 2020) . it was not until the middle of june that restrictions began to be lifted more generally. our analysis runs through to 15th june when many retail shops and public-facing businesses were allowed to re-open to the public. evidence regarding the impact of the lockdown rules on the use of greenspace is provided by community mobility reports (google 2020) . using data from mobile devices running google software enabled for location reporting, the mobility reports record changes in engagement in different activities over the lockdown period. the data is presented as a daily time series by region and records the percentage change in visits to numerous types of destination. our focus is on the data provided on trips to parks which google describe as including locations such as national parks, marinas, public beaches, dog parks, plazas and public gardens. google also comment that the parks data does not include visits to "the general outdoors found in rural areas" (google 2020) . this paper uses the google time series for 86 regions in england spanning the period 15th february to 7th june 2020. 4 , 5 each data point in a time series indicates the park 4 the regions used by google are level 2 administrative areas identified by the gadm (release 3.6) datab a se (global administrative areas 2020), which aligns very closely to counties and unitary authorities. 5 regional time series are not always complete. data points are missing when the numbers engaged in an activity on that day fall below google's privacy threshold such that there is insufficient data to ensure anonymity. no further information is provided by google on this censoring process and in the analyses that follow, we do not attempt to correct for the absence of these data points. visitation observed on that day relative to activity levels observed in that region over a baseline period. the baseline period used by google is the five weeks from 3rd january to 6th february 2020. in particular, a data point shows the percentage difference in visitations on that day relative to the median visitation observed for that same day of the week over the baseline period. throughout this paper we refer to that measure as one of park visitation change. the time series for england as a whole is shown in the top panel of fig. 1 , overlain with a smooth plot showing the central trend of the time series over the period. observe that the visitation change data initially oscillates around an average value of 6.1%. in other words, the park visitation measured by google over the period before the lockdown was around 6.1% higher than that measured over the baseline period. the impact of the enforcement of a strict lockdown on 23rd march appears to leave a clear signal. over the seven weeks from 23rd march through to 10th may visitation change falls to around 17% of baseline levels. likewise the relaxing of lockdown measures around 10th may, including the sanctioning of driving for engagement with the outdoors, coincides with a sharp upswing in parks visitation. on average over that last period of the time series visitation change is around 37% above baseline levels. on first examination, the google data appear to support the notion that outdoor recreation patterns in england were significantly affected by the lockdown rules. google, however, caution against over-interpretation of the raw data (google 2020) . the baseline for the data (3rd january to 6th february 2020) was chosen as a period before widespread disruption from covid-19. even without the disruption of covid-19 and the lockdown, we would expect outdoor recreation patterns to change from the winter months of the baseline to the spring and summer months of the lockdown. the central and bottom panels of fig. 1 over-plot the park visitation change time series with temperature and rainfall data for england. 6 on both panels, a smooth of the weather data is provided to identify the central trend. figure 1 reveals that the beginning of lockdown on march 23rd coincided with a well-defined change in the weather in england. after a very wet february and early march, the uk entered a prolonged dry spell. temperatures also began to increase, starting in the low tens at the beginning of lockdown and climbing to the low twenties by the end of may. a reasonable expectation might be that outdoor recreation would increase with that warmer and dryer weather, an expectation that runs contrary to the sharp fall observed in the park visitation change time series at the beginning of the lockdown. after an initial sharp fall, the visitation change data assumes a general rising trend that mirrors the rising temperature across england. it would be reasonable to assume that at least part of the differences in visitation seen over this period are attributable to the improving weather. in a similar vein, it is evident that visits respond to particular weather events. down spikes in the google data can be seen to coincide with significant rain events. likewise some of the peaks in the visitation data appear to correlate with spells of warm weather. in this paper, we take the patterns of change as suggesting that the story of greenspace use under lockdown in england can be broadly characterised as consisting of two distinct periods; • strict lockdown rules (23rd march to 10th may) over the first period of lockdown the restrictions on the use of greenspace will have exerted downward pressure on recreation activity. we expect also that behavioural adjustments to avoid infection over this period will have further reduced demand relative to normal activity levels. the upward trend in visitation change after the initial sharp fall, may reflect improving weather conditions. • relaxed lockdown rules (11th may to 15th june) entering this second period of lockdown, two things changed. first the rate of new cases had begun to fall, suggesting that england was past the peak of the virus and that the risk of infection was now falling, perhaps more significantly restrictions on outdoor recreation were lifted. both those factors will have acted to increase visitation to outdoor greenspace. that these increases in visitation are so substantial suggests that demand for greenspace may also have been inflated by the lack of alternative uses of leisure time coupled with a large segment of the population being freed from the time constraints of their normal working conditions 7 the orval model is underpinned by the orval greenspace map, a detailed spatial dataset that describes the location and characteristics of accessible greenspace across england (day and smith 2016) . the orval greenspace map identifies some 128,295 greenspace sites in england that could form the focus of a recreational trip. each recreation site is described by its physical characteristics including its dimensions, landcovers, designations and points of interest. data to estimate the orval model was provided by the monitor of engagement with the natural environment (mene) survey (natural england 2017). collected for the purposes of uk government national statistics, the mene survey provides a large, representative and random-location sample of adult (over 16 years of age) residents of england. the survey records trips to greenspace taken by each respondent over the seven days prior to the interview. for one randomly selected trip, the focus trip, the survey elicits detailed information including the location of the site visited and the mode of travel used to reach that destination. the mene survey runs throughout the year, sampling at least 800 respondents each week ensuring the data is temporally representative. orval was estimated from seven waves of data from 2009/10 through to 2015/16. in estimating orval, the destinations of focus trips in the mene data were matched to the orval greenspace map and choicebased sampling used to draw 78,154 observations for the purposes of model estimation. our econometric estimation corrects subsequently for the nature of the sample selection rule (manski and lerman 1977). 8 given the nature of the mene data, the orval model progresses from the assumption that each day represents a recreation choice occasion on which individuals can select from a choice set comprising (1) not taking an outdoor trip, and then (2) an option for traveling to each site by car and (3) an option for each site visited on foot. as such, our econometric model takes the form of a repeated discrete-choice recreation demand model (morey et al. 1993; breffle and morey 2000) where the repetition is over recreation decisions each day and the discrete choice is the decision over which of the options to select from the choice set. one significant complication in estimating a recreation demand model for all recreation possibilities across an entire nation is the size of the choice set. in estimating the orval model we make use of techniques of importance sampling to select a choice set for each individual that provides us with reasonable power in identifying the parameters of the model (guevara and ben-akiva 2013). our subsequent estimating procedures make corrections for choice-set sampling (daly et al. 2014) . following standard practice the orval model is constructed from a linear specification of conditional indirect utility functions (mcfadden 1973) . for the option of not taking a trip to an outdoor recreation area (alternatively, to choose the outside good) utility is assumed to be a function of an individual's characteristics (e.g., age, ethnicity, dog ownership, gender) the features of the particular day (e.g., the weather, time of the year, day of the week) and a set of spatial fixed effects defined by administrative regions at the level counties, unitary authorities and london boroughs. more formally, the utility of the outside good, labelled option 0, for person i on day t , is given by; where v i0t is the modelled part of utility which is taken to be a linear function of the factors assumed to influence choice of the outside good, labelled x it , and a set of parameters, 0 . finally, i0t is an econometric error term. a similar formulation is used to characterise options where recreation is chosen. these options are two-dimensional; they comprise both the choice of a greenspace destination and a mode of transport. in the orval model we assume that the utility from a site-mode combination is driven by two main factors; that site's characteristics including its landcover (e.g., woodland, natural grass, saltmarsh), designations (e.g., national park, country park, nature reserve), points of interest (e.g., archaeological remains, historic buildings, playgrounds, car parking facilities) and, second the costs that the individual incurs in travelling to that site by a particular transport mode. in orval those calculations are expressed as a monetary travel cost, tc ijq ; that is to say the combined costs in time and money that footnote 8 (continued) number of observations in each category. following manski, and lerman (1977) we correct for choicebased sampling through reweighting observation in the log-likelihood where the weight for observations in a category are simply the ratio of the population share making that choice to the same share in the sample. individual i incurs in traveling to site j using mode q (i.e. car or walk). 9 accordingly our model of site-mode utility is given by; where v ijqt is modelled utility for a site-mode option which is a linear function of a vector of site characteristics, labelled z j , associated with a set of parameters, 1 . utility is also determined by the travel costs of that site-mode option, tc ijq with associated parameter interpretable as the marginal utility of income. again, ijqt is an econometric error term. our estimating equations follow from the choice of distribution for the error terms, i0t (∀i, t) and ijqt (∀i, j, q, t) . in the orval model we assume those errors are draws from a distribution in the generalised extreme value (gev) family (mcfadden 1978) . more specifically, we assume that the errors are independent over individuals (i) and time (t) while allowing for the possibility of correlation in error terms across site-mode options belonging to the same, pre-defined similarity group. in orval, those similarity groups are identified by mode of transport (i.e. car, walk), the type of recreation site (i.e. park, path, beach) and the landcovers and land uses characterising a site (i.e. agriculture, allotment, church yard, moors and heath, natural grass, coastal, woods, wetlands, managed grass and fresh water). site-mode options can be members of more than one group, with the degree of membership of an option in a landcover group being determined by the proportion of a site's area under that landcover. a final, single-member group contains the outside option. those particular assumptions lead us to the cross-nested logit model specification (bierlaire 2006) in which the probability of a particular mode-site option is given by; here p ijqt represents the probability that person i , chooses to visit site j using mode q in time period t . in eq. (3) similarity groups are indexed by n = 1, 2, … , n , jqn identifies the pre-determined membership of site-mode option j, q to similarity group n and n (n = 1, 2, … , n) are parameters that capture the level of correlation in error terms for members of group n. equation (3) can be developed into a likelihood function for the observed choices and the model parameters, 0 , 1 , and estimated through methods of maximum likelihood. a full description of the development of the orval model, the parameter estimates and robustness testing is available in day and smith (2017) . 10 (2) travel costs for driving and walking are calculated from each respondent's home address to each site through the ordnance survey's detailed road and path network for the uk using state-of-the-art optimal routing algorithms (dibbelt et al. 2016) . fuel consumption while driving was estimated using formulae provided by the uk department of transport (2014) for an average family car and converted to a cost by multiplying by the price of fuel current in the respondent's region in the month in which they were surveyed. driving and walking times were converted into costs following guidelines on the valuation of travel time provided by the department of transport (2015). 10 note that while day and smith (2017) is a report to the uk government, the orval model was developed under expert oversight and subjected to academic review (see research project website: defra 2018). given it is based on a spatially and socioeconomically representative sample, orval can be used in exercises predicting recreation activity for the population of england. estimating visits is relatively straightforward. given an individual's characteristics and their travel costs for each site-mode option, eq. (3) can be used to predict the probability of them visiting some particular site using a particular transport mode on a particular day. in the analyses we present later, our focus is on predicting the number of visits to a region over a particular period of time. to estimate that for an individual using the orval model, one would simply sum the daily probabilities of visiting a site in that region where the probabilities would differ from day to day over that period on account of changing weather, day of the week and month of the year. to estimate total visits to the region over that period one would sum the result of that calculation for all adult residents of england. the predictions reported in this paper make a number of simplifications to that calculation both to account for the availability of data and to manage the magnitude of the calculation task. first, our predictions are based on the populations of small-area statistical areas named lower super output areas (lsoas) in england. the socioeconomic characteristics of lsoa residents was taken from the 2011 census and augmented with 2016 population estimates. we identify the population in each lsoa falling into 8 discrete groups defined by two key drivers of recreation engagement; socioeconomic segment and dog ownership. 11 taking averages of other sociodemographics, allows us to calculate daily visitation probabilities by group and lsoa. to enable comparison with the observed google mobility data, we require orval visitation predictions not only for the period of lockdown under both strict and relaxed rules but also for the period used as a baseline for the google data; a total of 120 days. a second simplification we adopt in our analyses is to group days into categories and only estimate visitation probabilities for each category. in particular, we categorise days according to month and whether they fall on a weekday or a weekend. our prediction period spans 6 months giving a total of 12 such day-month categories. in making visitation predictions we then use the met office daily weather data (see sect. 3.2) to calculate the average weather experienced in each lsoa for every day-month category. our most disaggregate visitation probabilities, therefore, constitute predictions for each day-month category from a socioeconomic group in an lsoa to a recreation site. aggregation to regional visit estimates on a particular day-month combination proceeds through a number of steps. first, for each socioeconomic group in an lsoa, we sum the visitation probabilities for that day-month combination across all sites in a region. multiplying up by each group's population in that lsoa and summing provides an estimate of visitation from that lsoa to the region. repeating those calculations across each of the 32,844 lsoas in england and summing the results provides orval's estimate of visits to a region. since we will have cause to refer to this calculation later, a more formal presentation is given by; where v gmd is the orval estimate of visits to region g on the particular day-month combination given by the index md where m indexes months and d ∈ {weekday, weekend} ; r indexes lsoas while s indexes the set of socioeconomic groups, such that n s r is the number of individuals in group s living in lsoa r ; c g is the set of site-mode options in region g and p s r jqmd is the orval estimate of the group-day-month probability of visiting site j by transport mode q. one useful property of gev models is that there exists a simple closed-form expression for the expectation of the maximum utility a respondent might expect to derive from being able to choose an option from their choice set. in the case of the cross-nested logit model that expression amounts to; where w it (c) is the expectation of maximum utility realised by individual i in time period t given the opportunity to choose from the set of site-mode options in the choice set c , and is the euler-mascheroni constant (that takes a value of approximately 0.5772). it follows that the expected level of welfare change that an individual would experience if the nature of their choice set were to change can be estimated from (small and rosen 1981) ; where c is the original choice set and c ′ is the changed choice set. in simple terms, eq. (6) describes the analyst's best estimate of how an individuals' utility will change as a result of changes in the choice set with that quantity translated into money terms by dividing through by the marginal utility of income, . in this paper, the choice set restriction explored is the one created by the strict lockdown rules where individuals were prohibited from travelling to outdoor recreation sites by car. as with our visit calculations, arriving at welfare estimates for such changes for the whole of england requires aggregating up from group-day-month welfare estimates calculated at the lsoa scale. using eq. 4, we generate daily predictions of recreation activity over the lockdown, simulating the lockdown rules by removing the option of driving to greenspace from each individual's choice set over the period of strict lockdown rules and returning those options to the choice set over the period of relaxed lockdown rules. in order to draw comparison with the google mobility data, these orval predictions must be expressed in terms of visitation levels relative to the baseline period (3rd january to 6th february 2020). accordingly, we also estimate visitation to each region during the baseline period, quantites we denote v gd 0. 12 daily orval predictions of relative regional visitation, compatible to those in the google data can then be calculated according to v gmd ∕v gd 0 . figure 2 plots out these orval prediction of visitation change over the lockdown period comparing them to those in the google mobility data. in interpreting fig. 2 , it is worth noting some caveats regarding the validity of a straight comparison of the two data series. first, there is not perfect congruence in the set of locations considered as outdoor recreation destinations. google's estimates, for example, ignore recreational use of countryside paths, trips that are included in orval estimates. second google's data reports on visitors to regions irrespective of their home location while orval is restricted to visits from residents of england. third, orval predicts day trips to greenspace locations but the google data does not distinguish between day trips and trips made while staying overnight away from home. fourth, the google data records visits by individuals carrying mobile devices enabled for location reporting, a group which does not necessarily represent the adult population of england whose behaviour is modelled by orval. as a final comment, we note the fact that the google data is reported in relative terms. accordingly, our comparisons are predicated on the assumption that changes in recreation behaviour in the areas of incongruence between the two data series experience the same relative changes as those where they overlap. observe that a sharp step down in the orval predictions is evident as the strict lockdown rules are brought into force and the option of driving is removed from choice sets. the predicted time series steps up again when the recreation activity rules are relaxed and continues on to the date at which the general lockdown began to be lifted on 15th june. unfortunately, at the time of writing google had not released its mobility data for the period between 7th june and that date. 12 since the baseline spans 2 months we acquire four estimates of these region-visitation quantities; weekdays and weekends in january and february. given the baseline period comprises 29 days in january and only 6 in february, we reach an estimate of weekday and weekend visitation in the baseline, v gd 0 , as a weighted sum of the estimates from those 2 months. the parameters of the orval model are estimated from the observed recreation behaviour of the english population under normal conditions. the fact that over the strict lockdown period the orval predictions are relatively lower than the baseline arises, therefore, purely on account of the removal of the option to travel by car. the predictions do not make adjustment for the other possible drivers of visitation change under lockdown. all the same, the orval time series does a reasonable job at defining the central trend of the google data over this period. in the period of relaxed lockdown rules, the orval predictions rise to a level of around 10% above the baseline. again these predictions simply reflect normal recreation in may and june which tends to exceed that in the winter months of the baseline. notice, however, that over this second period of lockdown the orval predictions lie well below the central trend of the google data. clearly, the recreation behaviour recorded in the google mobility data over this period cannot be explained solely on account of normal variation in recreation activity across the year. a further clear pattern of difference between the google time series and the orval time series concerns recreation activity over weekends. in fig. 2 the saturday of each weekend is marked by a light grey vertical line. recall that both time series are expressed in measures of visitation relative to the baseline. accordingly, while orval predicts weekend rates of visitation to be substantially higher than midweek visitation, it does so both in the baseline period and in the periods of lockdown. indeed, for orval, the ratio of the weekday and weekend predictions to their counterparts in the baseline remain relatively constant for both lockdown periods. the same is not true of the google time series. following the commencement of lockdown, that data series is characterised by a regular pattern of down spikes coinciding with weekend periods. since those same down spikes are not evident in early march, they are suggestive of a systematic change in behaviour during the lockdown. in particular, lockdown appears to have resulted in a relative redistribution of visits across the week with comparatively more trips being taken on weekdays when compared to weekends. such changes are compatible with a relaxing of leisure time constraints amongst workers normally limited to weekend periods for their outdoor recreation. figure 2 makes clear that the use of greenspace over the lockdown was not simply normal patterns of recreation behaviour constrained by the lockdown rules. indeed, differences between the orval and the google time series provide insights into the scale of the demand shifts precipitated by the various other factors impacting on greenspace use over this period. accordingly, the next step in our analysis is to use those observed differences to estimate parameters for the orval model that capture the demand shifts experienced under lockdown. within the orval model, a demand shift parameter, ̃ , can be specified as a fixed factor entering the utility function for the outside good. adding that parameter to eq. 1 we get; if ̃ takes a negative (positive) value then the utility of the outside good falls (increases) and visiting greenspace is relatively more (less) attractive. of course our comparison of the google and orval time series suggests that the level of demand shift differs from the first period of lockdown to the second and, during each of those periods, from weekdays to weekends. accordingly, we seek to estimate four demand (7) v i0t = x it 0 +̃ shift parameters, ̃t d where d ∈ {weekday, weekend} and t indexes periods of the lockdown; that is, t ∈ t 1 , t 2 . we build our estimating equations from the basic assumption that, augmented by the true shift parameters, the orval model provides unbiased estimates of the daily visits to a region's greenspaces. recall from eq. 4 that to reduce computational burden, predictions of visitation on day t are approximated by an estimate specific to the month of t m t and whether t is midweek or on a weekend d t . the calibrated orval estimate of visitation to region g on day t , therefore, can be denoted v gm t d t ̃t t d t where t t indicates the period of lockdown in which day t falls. the actual number of visits, v gt , differs from the orval estimate on account of myriad factors that we relegate to a mean-zero error term. according to this model, the google and orval estimates of relative visitation to region g on day t of lockdown period t are related according to the equation; where y gt is the visitation change observed by google, v gt 0 is the median level of visitation to region g on the same day of the week as t during the baseline and v gd 0 t is orval's prediction of visits during the baseline on a day equivalent to that identified by d t . we progress by assuming that the error terms in eq. (8) are independent draws 13 from a mean-zero normal distribution with variance 2 . it follows that the right-hand-side of eq. 8 amounts to a ratio of normal variates with identical variance but different means. such a ratio is a cauchy distributed variate with probability density function p y y; 1 , 2 , 2 , where 1 is the mean of the normal variate in the numerator and 2 the mean of the normal variate in the denominator (see hinkley 1969 for the exact functional form of this probability). given values for the demand shift parameters and the variance parameter, 2 , therefore, we can calculate the probability of observing each data point in the google time series according to prob y gt |̃t the demand shift parameters can then be estimated by solving the maximum likelihood problem; where y is the vector of google parks visitation observations for each region over each day of the lockdown period and ̃ is the vector of demand shift parameters to be estimated. the possibility exists that behavioural responses to lockdown may have differed across england. to explore that possibility we expand eq. 9 into a latent class regression analysis (wedel and desarbo 1994) . in this analysis we assume that the english population consists of a finite set of unobserved sub-populations or classes, indexed by h = 1, … , h with each class characterised by different demand-shift parameters, ̃ h . the unobserved size of the population in each class is given by a group membership proportion h (with ∑ h h = 1). the log likelihood for the latent class regression is given by; where class membership probability, h , is specified as a function of a parameter h according to exp . the parameters to be estimated include the demand shift parameters for each class, ̃ h , the class membership parameters = 1 , … , h and the class variance parameters 2 = 2 1 , … , 2 h . following standard practice (nylund-gibson and choi 2018), the log likelihood in eq. 10 was maximised over a series of different assumptions regarding the number of classes, with a four-class model being chosen as the model delivering the best fit according to the bayes information criterion (bic). parameter estimates from that model are reported in table 1 . the a priori class membership probabilities, h , suggest a fairly even distribution of membership over the four classes ranging from 19.2% in class 3 up to 36.5% in class 2. to help in the interpretation of the demand-shift parameters, fig. 3 plots out the implied park visitation change time series associated with each different class. in that figure, comparison is made to the uncalibrated orval predictions; a time series which assumes that the only change experienced during the lockdown was the imposition of restrictions on recreation activity. the shaded areas show how demand for trips to the outdoors for each class differs from that reference level. areas shaded in green show periods where demand for trips to the outdoors exceeded the reference, those in red where demand fell below the reference. the first thing to note from fig. 3 is that for each class the demand-shift parameters distinguish a change in relative preferences for recreation on weekdays as compared to weekends. compared to the reference (orval's uncalibrated time series), over the lockdown relatively more trips are taken during the week and relatively less on weekends; possibly a result of an easing of leisure-time constraints on furloughed workers. also observe from fig. 3 that when the lockdown rules were relaxed, levels of demand for all four classes substantially exceed reference levels. that pattern possibly reflects a substitution effect as people turned to outdoor recreation in lieu of access to other prohibited leisure activities. it might also reflect an increasing propensity to engage in outdoor activities as the risks of infection diminished. considering the class 1 predictions, notice that over both periods of lockdown the time series exceeds that of the uncalibrated reference; the net effect of the demand shifters for this class is to increase use of the outdoors. indeed, class 1 represents the sub-population whose demand for the outdoors increased most substantially under lockdown. the patterns of recreation activity expressed by populations in class 2 and class 3 are reasonably similar. in both, over the period of strict lockdown rules, recreation activity tracks reference behaviour, differing primarily in the redistribution of visits from weekends to weekdays. that redistribution effect is somewhat more substantial for class 2 populations. over this first period of lockdown, it appears that for classes 2 and 3 the demandreducing effect of virus-exposure risk and the demand-increasing effect of restrictions on alternative leisure options are either small or act to cancel each other out. after the relaxation of lockdown rules, both classes exhibit a similar and substantial upward shift in demand for recreation, though the redistribution of trips from weekends to weekdays remains more pronounced in class 2. class 4 are the only population to exhibit levels of recreation activity than are consistently lower than the reference. for these populations the period of strict lockdown saw engagement with the outdoors fall below that which might be expected just from the restrictions on driving to recreation locations. after the relaxation of that rule, class 4 populations expanded their demand for outdoor recreation above reference behaviour, but considerably less so than the other populations. while the group membership probabilities of table 1 provide an indication of the mix of different behavioural classes across england, it is also possible to derive an estimate of the specific mix characterising visits to each region of the google data. using bayes theorem, the posterior probability that the observed visitation data for region g results from populations expressing the class h recreation pattern of recreation activity is; calculating such posterior probabilities for each class, we arrive at a set of estimates g h ; h = 1, … , h that we interpret as representing the proportions of visitors from each class contributing to recreation activity in region g. accordingly, we refer to those quantities as the class shares for a region's visits. our objective is to use these class shares to determine the class most likely to represent the recreation behaviour of the population of each lsoa. knowing those classes allows us to calibrate the orval model by assigning the appropriate demand-shift parameters to the choice equations for residents of each lsoa. orval can then be used to derive estimates of recreation activity and welfare changes under lockdown condiations. one approach to assigning classes to lsoas would be to identify the region in which an lsoa is located and ascribe it the class for that region with the highest visit share. the intuition here is that the majority of visits from an lsoa, r , will be to the region in which it is located, g r , such that our best guess of the behaviour class of an lsoa's population will be that most frequently observed in visits to g r . of course, that calculation ignores the fact that residents of an lsoa may also visit other regions, such that information about the behaviour class of an lsoa is also contained in the class shares of visits to those other regions. to make use of that information, we make an initial guess at the trips taken by residents of lsoa, r , to each region, g, 14 and use those to calculate the proportion of visits from r that choose g as a destination. using these proportions as weights, we calculate the weighted sum of the class shares for each region's visits, to arrive at our best guess of the class shares characterising r . we assign r the class exhibiting the highest class share. figure 4 maps out the classification of lsoas in england to different classes. to simplify presentation and reflect their similarity, areas in class 2 and 3 are presented in the same shade. while the data is plotted at the lsoa scale, as might be expected, the pattern in particular, we calculate the visits from each lsoa to each region assuming class 1 behaviour, then repeating those calculations for each of the three remaining behavioural classes. our estimate of visits from lsoa r to region g are calculated as the weighted sum of those four visit estimates where the weights are given by that region's class-visit shares, of class membership broadly follows the regions upon which the data analysis is based. those regions are outlined in white and close inspection reveals that our classification procedure allots some lsoas along region borders to a different behaviour class to lsoas in the region interior. there exists some interpretable spatial pattern in the distribution of class membership described in fig. 4 . for instance, all the major metropolitan areas of england exhibit class 2 and 3 behaviour changes (expected activity under strict lockdown, much increased activity under relaxed lockdown). in addition, class 1 behaviour changes (increased activity under strict lockdown, greatly increased activity under relaxed lockdown), show clear patterns of regional clustering most notably along the south coast and central-south region of england. we suspect that these patterns reflect regional differences in the perceived and actual risks of exposure to the virus. areas exhibiting class 4 behaviour changes (reduced activity under strict lockdown, increased activity under relaxed lockdown) are largely located in relatively remote and rural areas of england. that pattern would be commensurate with locations whose workforces are primarily engaged in the food production sector; an occupation classed as essential in the lockdown and not subject to restriction under the lockdown rules. the top left panel of fig. 5 presents orval's predictions of visitation change for england once the recreational choices of residents of each lsoa have been adjusted with the demand shifters for their estimated behaviour class. applying the methods described in sect. 4.2, we can now use this calibrated version of the orval model to estimate levels of table 2 . the estimates in table 2 are for visits and values aggregated over all english residents over (1) the seven weeks of strict lockdown rules, (2) the five weeks of relaxed lockdown consider first the changes in estimated visits. where normally we would expect some 168.8 million trips to the outdoors taken by car, such trips were prohibited over the 7 weeks of strict lockdown rules. what the orval estimates reveal is that individuals responded to those restrictions through substituting to trips taken on foot. in the period of strict lockdown, the calibrated model estimates that 275.0 million trips were taken to greenspaces on foot, an almost 40% increase over the 198.8 million expected under normal conditions. figure 6 illustrates how recreation behaviour changed across england in this period. the left-hand panel plots out orval estimates of the spatial distribution of weekly visits taken by residents of the major metropolitan areas of england under normal conditions. the right hand panel contrasts that with the distribution of visits under the strict lockdown rules. prohibited from driving, outdoor recreation activity refocused on local greenspaces. once the restrictions on driving were lifted, the effects of the demand-shifts evident in the google data become clear. visits by both car and on foot increase, resulting in levels of recreation visits that are some 27.5% above those expected under normal conditions. the story of outdoor recreation under lockdown is one in which people offset the restrictions on driving to recreation sites by switching to walking to greenspaces local to their homes. that behaviour along with an upward shift in demand for recreation resulted in the overall number of visits to the outdoors over the lockdown period being little changed from that under normal restrictions. our calculations of welfare change suggest that the cost of lockdown on welfare derived from greenspaces was negligible, dropping by £150 million or some 2.7% of that realised under normal conditions. a second set of analyses that are possible with the orval model use the calibration parameters to explore the visit changes and welfare consequences that might have arisen should alternative rules on recreation have been instituted in the lockdown. here we consider two such counterfactuals. the first is a counterfactual where the strict lockdown rules prohibiting driving to greenspaces were extended over the whole period from march 23rd to june 15th. the second is a counterfactual in which no restrictions were imposed on recreation activity over lockdown. time series describing recreation activity under those two counterfactuals are presented in the bottom panels of fig. 5 . observe that in the strict lockdown counterfactual we are projecting behaviour out over the second period of the lockdown under rules for which we do not have observations from the google mobility data against which to calibrate. two assumptions are possible. first that the demand shift parameters characterising behaviour under the strict lockdown period continue to characterise behaviour under the extension of those rules into the second period. alternatively, that the second period might be characterised by the demand shift parameters charactering recreation behaviour during that second period under the relaxed rules. since the demand shift parameters of the second period are universally more positive than those for the first period, those two assumptions suggest lower and upper bound estimates of possible behaviour under the strict lockdown counterfactual. those bounds are traced out in the plot of fig. 5 with the grey shaded area demarking the paths lying between those bounds. similar arguments lead to bounds on the recreation activity over the first period in the relaxed lockdown rules counterfactual. these too are shown in fig. 5 . summary details of recreation visits and values under the strict lockdown counterfactual are presented in table 3 . in that table, we present estimates that are averages of those for the lower and upper bounds and contrast those with estimates of visits under the lockdown under the actual lockdown rules. not surprisingly, maintaining the rule prohibiting driving to outdoor recreation locations has the effect of suppressing engagement with greenspaces. the orval model predicts some expansion of walking in the second period of lockdown to compensate for the continuing restrictions on driving opportunities. all the same, maintaining strict rules on recreation over the whole lockdown results in an estimated 18.4% reduction in visits to the outdoors compared to those estimated under the actual lockdown rules. in terms of welfare, the stricter rules impose a welfare cost on english residents; the value flow realised from greenspace access falls by some £1.14 billion. viewed the other way, the government's decision to relax the rules on outdoor recreation activity delivered a £1.14 billion welfare boost to residents of england. table 4 provides an identical analysis for outcomes under the relaxed rules counterfactual in which the lockdown proceeded without restrictions on outdoor recreation activity. under the relaxed-rules counterfactual orval predicts an expansion of recreation activity. visits to the outdoors are some 24.2% greater than those estimated under the actual lockdown rules. again that translates into changes in the economic value of greenspace. a lockdown with no restrictions on recreation activity increases the estimates of the welfare benefits of greenspace access by some £1.47 billion. viewed the other way, english residents suffered a welfare cost of £1.47 billion as a consequence of the government's decision to restrict recreation activity over the first period of the covid-19 lockdown. using analytical methods that leverage google mobility data and the predictive powers of the orval model, this paper explores how the covid-19 lockdown in england changed how people engaged with greenspace and impacted on the economic value they derived from those interactions. we find strong evidence to support the contention that greenspace became a significant source of welfare for citizens at a time when opportunities for alternative uses of leisure time were even more seriously curtailed. one key change identified by our analysis is that the lockdown rules forced citizens to get out of their cars and walk. trips to greenspaces by car fell by 47% over the whole lockdown period with an attendant 34% rise in trips taken on foot. increased engagement in outdoor recreation (particularly in the second period of lockdown) coupled with this substitution behaviour meant that, despite the restrictions citizens maintained value flows from greenspace over the lockdown comparable to those they would have enjoyed over that same period under normal conditions. our analysis also explores how the welfare derived from greenspace might have differed under alternative lockdown rules. we discover that the adoption of more relaxed rules on the use of greenspace during the first period of the lockdown would have delivered increased welfare flow from greenspace of £1.47 billion. a retrospective interpretation of the decision to impose limitations on engagement with greenspace, therefore, would be that the government judged that the health costs associated with the increased risk of infection from adopting less strict rules over that period were in excess of £1.47 billion. a second counterfactual policy considered the maintenance of the rules limiting engagement with greenspace into the second part of the lockdown. our analysis reveals that such a policy would have reduced the value flow from greenspace by £1.14 billion. the retrospective interpretation of that figure is that by the time the rules on outdoor recreation were relaxed the government judged that the societal costs of the increased infections that might arise as a consequence, to be less than £1.14 billion. several important research questions remain to be answered and the analytical framework developed in this paper stands well placed to address them. as our analysis reveals, behavioural responses to the lockdown differed across the country. in this paper we offer only tentative speculations as to why those differences arose. a more detailed analysis relating the observed changes in outdoor recreation activity to factors including regional differences in the risk of exposure to covid-19, profiles of occupations, sociodemographics and the local availability and quality of greenspaces might reveal important information as to the key drivers of outdoor recreation behaviour under lockdown. likewise that detailed exploration of spatial differences in outdoor recreation activity, might help identify those communities that were most seriously disadvantaged by the lockdown restrictions perhaps on account of the lack of availability of high quality local greenspace. our analysis also reveals that in the second period of lockdown, use of the outdoors expanded very substantially, far exceeding that expected under normal conditions. the google mobility data accessed for the purposes of this analysis provided observations only as far as 7th june 2020. more recent data releases suggest that this increased demand has been maintained even as other areas of everyday life gradually return to normal. that trend has led to speculation that the covid-19 lockdown has precipitated widespread "re-engagement" with outdoor recreation and is perhaps evidence of a structural shift in preferences for greenspaces (royal society for the protection of birds 2020). revisiting the google mobility data in a few months' time and extending the analyses of this paper should help establish the degree of persistence of that shift. if covid-19 has indeed led citizens of england to discover the delights of the outdoors then perhaps that offers a faint glimmer of positive news in a period so scarred by suffering. open access this article is licensed under a creative commons attribution 4.0 international license, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the creative commons licence, and indicate if changes were made. the images or other third party material in this article are included in the article's creative commons licence, unless indicated otherwise in a credit line to the material. if material is not included in the article's creative commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. to view a copy of this licence, visit http://creat iveco mmons .org/licen ses/by/4.0/. haunting pictures show weston-super-mare in lockdown a theoretical analysis of the cross-nested logit model investigating preference heterogeneity in a repeated discrete-choice recreation demand model of atlantic salmon fishing seaside resorts at risk of becoming ghost towns due to coronavirus. the telegraph practical solutions for sampling alternatives in large-scale models the outdoor recreation valuation (orval) tool: data set construction expanding and improving the outdoor recreation valuation tool-nr0168 values of time and vehicle operating costs: tag unit 3.5.6. department for transport, london department for transport (2015) provision of market research for value of travel time savings and reliability customizable contraction hierarchies measuring the benefits of water pollution abatement covid-19 community mobility reports sampling of alternatives in multivariate extreme value (mev) models 85/covid -19upd ate covid-19) statistics: hmrc data about the coronavirus job retention scheme, the self-employment income support scheme, and the vat payments deferral scheme the green book: central government guidance on appraisal and evaluation on the ratio of two correlated normal random variables coronavirus: empty streets in brighton during lockdown. the argus prime minister's statement on coronavirus (covid-19) a model of weekend recreation travel demand here-are-the-new-rules -as-newso m-order s-all-calif ornia ns-to-stay-at-home the estimation of choice probabilities from choice based samples modelling the choice of residential location. spatial interaction theory and planning models midas: uk daily temperature data. ncas british atmospheric data centre. https :// catal ogue.ceda.ac.uk/uuid/1bb47 9d3b1 e38c3 39adb 9c82c 15579 d8 midas: uk daily rainfall data a repeated nested-logit model of atlantic salmon fishing monitor of engagement with the natural environment: the national survey on people and the natural environment ten frequently asked questions about latent class analysis understanding drivers of change in park visitation during the covid-19 pandemic: a spatial application of big data recovering together a report of public opinion on the role and importance of nature during and in our recovery from the coronavirus crisis in england applied welfare economics with discrete choice models urban nature in a time of crisis: recreational use of green space increases during the covid-19 outbreak in oslo a review of recent developments in latent class regression models. in: bagozzi r (ed) advanced methods of marketing research key: cord-339309-r70zd30q authors: havell, richard; jenkins, chris; rutt, james; scanlon, elliott; tregear, paul; walker, mike title: recent developments at the cma: 2019–2020 date: 2020-10-06 journal: rev ind organ doi: 10.1007/s11151-020-09790-y sha: doc_id: 339309 cord_uid: r70zd30q we discuss three important cases that the competition and markets authority (cma) has completed over the past year: first, the coronavirus pandemic has had implications for a wide range of the cma’s work; we describe the work on price gouging conducted by the cma’s covid-19 taskforce and respond to the argument that competition authorities should not be concerned about such behaviour. second, a number of high-profile studies have considered the appropriate application of competition policy in digital industries. the second two cases—the online platforms and digital advertising market study, and the google/looker merger—show the work the cma has continued to do in this area. the uk competition and markets authority (cma) is the uk's primary competition authority. this article discusses three important cases for the cma over the last year. the first case illustrates the cma's response to the coronavirus pandemic. the pandemic has fundamentally altered everyone's lives and has had implications for a wide range of the cma's work. in march 2020 the cma established a covid-19 taskforce to focus on the competition implications that have arisen from the pandemic. in march there were prominent reports of panic buying and there have subsequently been numerous media articles that have highlighted significant price rises for products such as hand sanitiser that have suddenly become essentials. this article discusses how the cma has responded to complaints about 'price gouging' and the role that economics has played in this work. along the way we address the important question: is this simply an efficient response to the sudden scarcity of certain products? the second two cases-the online platforms and digital advertising market study, and the google/looker merger case-illustrate the cma's ongoing work in relation to digital markets. the appropriate use of antitrust tools in digital markets has been the focus of a number of high-profile reports over the last two years. 1 in the uk the furman review 2 advocated for the establishment of a digital markets unit to support greater competition and consumer choice in digital markets and expressed concerns about underenforcement in mergers involving digital firms. the cma is in the process of providing advice to the uk government on the implementation of the furman review's recommendations, 3 and a number of countries have proposed new regulatory regimes. 4 the online platforms and digital advertising market study was a wide-ranging and comprehensive study that focussed on various aspects of digital advertising and in particular google's role in (i) online search and (ii) open display advertising and facebook's role in social media. the study considered the sources of market power for these two firms and identified how they are able to exploit that market power. the google/looker case demonstrates the ability and willingness of the cma to scrutinise acquisitions by leading tech companies and to respond to concerns of underenforcement in this area. the cma established a taskforce in march 2020 to respond to the novel competition and consumer protection challenges that were raised by the covid-19 outbreak. one of the issues from the beginning was the appropriate response to the huge numbers of complaints of sharply increased prices for important products. 5 other issues, which are not addressed here, related to advice on government support to industries and exclusion orders that provided temporary exemptions from competition law provisions. between 10 march and 28 june 2020, the cma received over 7,600 complaints about businesses that charged high prices. the volume of these complaints peaked in late march, and gradually declined from then on. 6 the majority of the price-based complaints related to what can be termed essential items, such as: toilet paper, meat, hand sanitiser, flour, rice, and eggs. the large majority of these complaints to the cma related to independent bricks-and-mortar retailers: a type of business which would not normally be considered to possess significant market power. 7 this section sets out the cma's economic assessment of the complaints in the exceptional circumstances of the covid-19 pandemic, through a competition lens. the crisis immediately changed shopping behaviour with consumers' shopping patterns becoming more local and a shift to greater purchases in the local independent outlets that were the subject of the majority of the complaints. the first step was to establish an online form for the complaints in which the product specification and outlet were recorded along with the pre-covid price and the high covid-period price being complained of. from these data, the median price increase complained of was around 160% (fig. 1 ). this varied widely by product, with hand sanitiser seeing median reported price increases of approximately 400%-a quintupling in price. whether short-term higher prices in an emergency ('price gouging') should be seen as an abusive practice is a controversial matter among economists. 8 the uk, along with many other countries, does not have specific legal provisions that are targeted 5 hayter (2020) gives an overview of the work of the cma during the covid-19 outbreak. 6 a large proportion of the non-price related complaints related to difficulties in obtaining refunds from businesses such as wedding venues and holiday cottages. 7 there were also complaints that were related to listings on online marketplace platforms-especially those for masks and other kinds of personal protection equipment (ppe). 8 among the chicago booth igm forum (2012) of prominent economists, only 8% agreed with a proposed anti-price gouging law. concerns with respect to efficient allocation and supply response were cited as the most common reasons for disagreement. see https ://www.igmch icago .org/surve ys/price -gougi ng/. at such conduct, which means that this conduct often falls to be considered as a variation of the abuse of excessive pricing. typically price increases are an important signal to induce a supply response while acting to ration demand to those with higher valuations for the goods and services. the concern that enforcement action will undermine the appropriate market responses informs the economic debate on appropriate conditions on excessive pricing. 9 the cma's experience highlights where, in exceptional crisis circumstances, prices may be exploitative and unfair, and appropriate competition authority action can safeguard vulnerable consumers without undermining supplier responses. there are good reasons why temporary price spikes during crises may not incentivise supply, and may reflect an exercise of market power to exploit consumers who have particularly poor alternatives. in this, it is important to distinguish between broadbased price changes that reflect supply-side factors, and very high price increases that are well in excess of those of comparable firms, by small numbers of retailers. such price increases are unlikely to pass useful signals to manufacturers where they reflect the extraction of large profits from sales without leading to retailers' paying higher prices to manufactures. fletcher (2020) points out that sharp price increases may be unnecessary in encouraging increases in output even when they do reach manufacturers. manufacturers may have limited ability in the short run to react to short-run and transitory price increases; and where there is flexibility in output the imbalance between supply and demand may be signalled without the need for a large price increase: for example, through an increase in the volume that is being ordered by customers. where firms engage in conduct such as strategic hoarding and reselling-and thereby divert goods from their normal supply chains-it is particularly likely that high prices will result without useful market signals' being produced. in addition, price spikes can be driven by consumers who are stockpiling goods. such short-run "demand spirals"-where customers faced with uncertainty around future supply choose to buy products when they have the opportunity to do so-do not reflect an increase in demand beyond the very short run. where such consumer stockpiling leads to higher short-run demand and higher prices, firms will anticipate depressed demand following the price spike as consumers use up their stocks, and so will not invest to increase output during this period. in the case of ordinary household goods that saw no increase in consumption, or small increases due to demand diversion towards in-home consumption, due to the pandemic there is little useful information that is contained in the signal of a very high price. another function of market prices is to ensure that scarce products are allocated efficiently. however, individuals do not vary widely in the inherent value that they derive from essential goods such as food. differences in the ability and willingness to pay high prices for essential goods will reflect the incomes of consumers more than differences in level of need. such inequitable outcomes will be made more severe by the effects of stockpiling, with the ability to stockpile at high prices being further limited to richer consumers. high prices may discourage stockpiling; but consumers who face uncertainty may see high prices as a signal of scarcity, which encourages stockpiling. 10 next we explain how these possibilities were tested by the cma in the actual experience of the covid-19 pandemic. price increases are less likely to serve a useful function in a market, and are more likely to exacerbate poor allocation outcomes without leading to improvements in the timeliness of supply responses, where the firms charging high prices have market power. the circumstances of the covid-19 outbreak mean consumers' alternatives have substantially narrowed meaning some local outlets have the potential profitably to raise prices substantially relative to other outlets such as national supermarket chains. this is quite different from higher prices for flights at mid-term, or cinema seats on a friday evening, which reflects inter-temporal price discrimination consistent with competition not the exertion of market power. these narrower geographic markets give greater market power to independent retailers in these local areas and, in particular, weaken the constraint on their pricing fig. 1 median reported price increases in complaints. note: this chart is based on 1,969 complaints that provide information on prices both before and after the outbreak, and shows only products with at least 60 complaints that is imposed by supermarkets. 11 this is especially the case for consumers who rely on public transport-which has also seen its capacity greatly reduced-and for more vulnerable consumers who are less able to stand in queues. another effect of social distancing was a reduction in the customer capacity of larger supermarkets, with many supermarkets requiring long queues to enter and many consumers avoiding larger stores to reduce their risk of exposure to the virus. aside from this, consumers' search costs are greatly increased during the outbreak. visiting multiple retailers to check the prices they offer is far more costly than normal: due to the additional time required and also the risk of exposure to the virus. these high search costs are compounded by widespread shortages, or perceived shortages, of essential products. customers will also be unable to observe whether potentially competing retailers have in stock the product that the consumers wish to buy. if a retailer is indeed out of stock, then it will represent no competitive constraint on other retailers-even if it would normally be a strong constraint. close examination of the physical retailers with the most complaints made against them find that they are generally located in the most socially and economically disadvantaged areas of the uk. this is consistent with their possessing market power temporarily accrued through the mechanisms described above. given that essential products account for a larger proportion of the spending of low-income consumers, the welfare harm that is associated with the price increases is larger for these consumers. the cma's analysis pointed to the local market issues and underpinned advice that related to the concerns and requests for information from identified outlets with regard to their pricing and the justifications for it. in addition, the cma engaged with owners of the major symbol groups, 12 who contacted their franchisees and referred to the cma's attention and the potential negative reputation effects of exploiting a short-term advantage. monitoring of the complaints and prices by channel confirmed that prices adjusted towards normal levels and no further action was taken, aside from the special case of hand sanitiser. hand sanitiser complaints stood out in terms of the large volume received by the cma-over 1,800-and the magnitude of the price increases complained of. these considerations-along with the importance of hand sanitiser in the context of the pandemic following government and scientific advice-led the cma to focus attention on it. a number of other countries responded to similar price hikes by simply capping retail prices of hand sanitiser. 13 ensuring its efficient supply is clearly 13 france, spain, and india all imposed caps on the prices of some hand sanitiser products. 11 the cma's 2017 investigation of the merger between tesco and booker found that supermarkets represent a strong competitive constraint on independent grocery retailers during normal times. this merger is discussed in basso et al. (2018) . 12 symbol groups are franchises of independently-managed grocery retailers that share a logo. examples of uk symbol groups are spar and costcutter. important to mitigating the spread of the virus; and given the strong positive externalities that surround its use, efficient allocation must include its widespread availability to consumers of all income levels. the cma's approach involved assessing the supply chain by contacting a number of participants in the hand sanitiser industry, including manufacturers, national chemists' chains, and symbol group owners. this complemented the analysis of electronic point of sale (epos) data. 14 the sustained higher demand levels led to volumes of hand sanitiser purchased by consumers increasing during the covid-19 outbreak that were approximately 15 times the pre-covid levels. 15 unlike most other products, hand sanitiser has therefore required a major supply response to meet demand. the cma's analysis sought to distinguish between pricing that is consistent with the supply response and pricing by some outlets that apparently reflected exploitative conduct at the retail level. the largest increase in sales of hand sanitiser has been in the grocery multiples retailer channel, which includes large supermarkets and the boots and superdrug chains-see fig. 2 . chemists (pharmacies) also saw a large proportional increase in the volumes of hand sanitiser that were sold. however, sharp decreases in the volumes sold through grocery multiples may be seen at certain times, including at the start of the social distancing regulations at the end of march. this is consistent with customers' being left with little choice but to turn to alternative retailers at this time, including independent grocery retailers and chemists. figure 3 plots the prices that were reported in complaints to the cma for 50 ml containers of hand sanitiser against the average prices that were observed in each of four retail channels. average prices in supermarkets increased only slightly over time. average prices in symbol group retailers, independent grocery retailers, and chemists did increase substantially in mid-march and again in early april. many of the complaints that were made to the cma, however, involved price increases that were far in excess of those at the elevated levels in non-supermarket outlets. complaints about such prices persisted beyond the period that was required for supply through the main supermarket channel to ramp up. an assessment of the manufacturing of hand sanitiser confirmed that material costs had increased, including for the main input: ethanol. however, taking into account other production costs, these increases were consistent with an increase in producer prices of no more than 20-25%. major manufacturers indicated that they had not increased their wholesaler prices (or had increased them only slightly) and had not generally changed their recommended retail prices. the major producers of hand sanitisers ramped up supply quickly. however, the spike in demand still meant scarcity relative to demand in the short run. retailers and wholesalers that still had stocks of hand sanitiser, or who could acquire stock at short notice, were able to collect large rents on this stock. in addition, other sources-including imports-were directed to the local market. accordingly, up to 16% of the hand sanitiser sales recorded in the nielsen data relates to brands with little or no presence in the uk prior to the outbreak. while more expensive, the costs of airfreighted imports did not justify the price hikes that were observed in some of the complaints. the cma identified a set of retailers that had been reported multiple times for charging particularly high prices for hand sanitiser products, and that had maintained these prices for a sustained period of time. information requests were sent to these retailers, with the key information requested being the cost prices that were paid for the high-priced products-to ensure that the retailers were indeed setting high markups rather than passing along high costs-and the number of sales made at the high prices: to establish that they did indeed possess the market power necessary to make substantial sales at the high prices. formal cases of potential excessive pricing were opened in the case of some retailers. in almost all cases investigated by the cma, retailers voluntarily adjusted their prices to reflect a normal level of markup. additionally, the cma and the general pharmaceutical council released a joint letter to all registered pharmacies in the uk that emphasised that the charging of high prices by some pharmacies may damage the public's trust and confidence in pharmacies, and asking pharmacies to ensure that their prices for essential products are fair and do not reflect large increases in markups over normal levels. 16 the covid-19 outbreak required the cma to be flexible and responsive in analysing markets for essential products, understanding the reasons for the price hikes being complained of, and deciding on the appropriate steps to take. in the case of price gouging, this led to the cma identifying that the reported high prices were charged mainly by independent outlets. the increases observed in complaints were far in excess of the average increases by comparable retailers, as well as being much higher than the averages that were observed across all retail channels including the main national supermarket and pharmacy chains. higher-than-usual prices are generally a signal for increased supply. however, the very high prices being observed were not reflective of the changes in average retail prices nor of increases at the producer level. the increases reflected two main factors at work: first, the impact of covid on shopping patterns meant that consumers had much narrower alternatives than was normally the case and local outlets had market power-at least temporarily. this was especially the case in areas that were already relatively poorly served by main supermarket chains-that happen also to be relatively low-income urban areas with less car ownership. it is in line with concerns that the cma has found in earlier matters such as the tesco-booker merger, where the markets that were served by local convenience stores were scrutinised. darker dots indicate multiple complaints made on the same day at similar prices. three complaints of very high prices-between £19 and £30-are not shown to avoid compressing the chart. average channel prices are based on nielsen data. grocery multiples includes the major uk supermarkets, including boots and superdrug; symbol group retailers includes retailers which are members of uk symbol groups; independents includes independent grocery retailers and convenience stores; chemists includes independent pharmacies and chemist chains second, simply allowing prices to spiral upwards to clear the market risks fuelling panic buying. anxious wealthier consumers may purchase all of the stock at the hiked prices while others, who may need products such as hand sanitiser even more, are faced with empty shelves. the result is not a socially optimal allocation of the product, but simply an inability of the less well-off to acquire it. in such circumstances, quantity rationing-restrictions on the number of the same item a consumer can purchase-rather than rationing on the basis of ability to pay, is likely to lead to better outcomes. some retailers, that are concerned about their longer-term reputation, have implemented such quantity rationing. the cma's role has effectively been one of market monitoring and assessment. simply analysing the complaints and following-up with retailers has brought about price adjustments. other market participants, such as symbol groups, and industry regulators have highlighted the harm from opportunistic price hikes. in other cases, the cma has engaged informally with firms and has encouraged price reductions for goods that were reported in complaints. through these means, the cma has aimed to maintain public trust in markets by preventing the most severe abuses while taking care to not disrupt supply responses. it also contrasts with countries where there has been direct regulation of prices of some products. the experience highlights the challenges to competition authorities in responding flexibly and at speed and the importance of understanding the nuances of how markets work in practice, and how consumers make choices as circumstances change. distribution of the percentage of google search events that were for queries that were seen by bing, and vice versa, by the frequency of their search query. source: cma analysis of google and bing data. notes: we define the head as the 15% of queries seen most often in a dataset and the tail as the 30% of queries seen least often in july 2020, the cma published the final report of its market study of online platforms and digital advertising in the uk. the study was launched in the context of concerns-that have been raised in the uk and globally-about the market power that is held by a small number of online platforms. it investigated a range of issues and aimed to inform the broader debate on the regulation of online platforms, as explored in the furman 17 and stigler center 18 reviews. these reviews concluded that relying solely on existing competition law was not sufficient and that a new procompetition approach should be taken to regulating platforms. these reviews also identified a number of themes that were central to the cma's market study: one is that in many cases, digital markets are subject to 'tipping' in which a winner takes most of the market. this results in highly concentrated markets, which may ultimately lead to consumer harm. firms with market power may raise effective prices or innovate less, as they have less to fear from new entrants. another is the trend for large platforms to create ecosystems of multiple services that retain consumer attention and harvest valuable data. this trend can be harmful if it provides multi-market firms with opportunities to leverage a strategic gateway position in one market to other adjacent markets, by self-preferencing or exploiting an advantage in data. recent literature has also highlighted possible 'platform envelopment' theories of harm, where expansion into adjacent markets can protect a platform's market power in its core market. 19 we do not attempt to report on all of the issues that are covered by the study here. instead we focus on the main components of the cma's analysis on the sources of market power held by online platforms and how this market power affects competition in the markets for digital advertising. first we outline the sources of google's market power in search and how it has exploited this market power when selling search advertising. 20 we then discuss the cma's analysis of google's role in the open display market, where intermediaries provide various technologies that allow online publishers to sell advertising inventory and advertisers to buy it. this includes both the way google has leveraged its strong position in its wider set of interlinked online services (its 'ecosystem') into the open display market and the potential for google to exploit its position on both sides of the intermediation chain to self-preference its own activities, thereby reinforcing its market power. finally, we look at the sources of facebook's market power in social media and how it has exploited this market power. the cma found that google has significant and enduring market power in both search and search advertising. google has had around 90% or more of the search market in the uk for over 10 years and accounts for 97% of searches on mobile devices. 21 bing (owned by microsoft) is the only other at-scale search provider in the uk. this finding is consistent with the results of several other investigations, such as the accc digital platforms inquiry 22 and various european commission investigations. 23 innovation played an important role in google's initial growth. however, scale has now become very important in search. strong network effects, economies of scale, and default positions 24 reinforce google's position and act as a barrier to entry and expansion for rivals. the relevance of search results is the most important dimension of search quality. one of the key inputs to producing more relevant search results is 'click-andquery' data: information on the search queries received by a search engine and how consumers have interacted with the results that they were served. search engines use click-and-query datasets to improve their algorithms and return more relevant results. click-and-query data are subject to network effects: as the search engine acquires more users and hence more click-and-query signals, the quality of the service for other users increases. the cma found that the marginal benefit of additional clickand-query data is higher for uncommon queries-which are also known as 'tail queries'. it analysed all of the 3-4 billion 25 search events seen by google and bing in a one-week period in the uk (fig. 4) . google saw 16 times more distinct queries than did bing in this period. as shown in the figure below, google saw around 30% of bing's tail queries, whereas bing saw only around 1% of google's tail queries. this demonstrates google's advantage in being able to serve more relevant results to uncommon queries and reinforces consumers' perceptions of google as the highestquality search engine. 21 all figures are for the uk. shares for ten-year time series sourced from statcounter. shares in mobile search are from cma analysis of parties' data and relate to december 2019. 22 accc (2019). 23 case at. 39,740 (google search (shopping)). see press release available at https ://ec.europ a.eu/ commi ssion /press corne r/detai l/en/ip_17_1784. case at.40099 (google android). see press release available at https ://ec.europ a.eu/commi ssion /press corne r/detai l/en/ip_18_4581. case at. 40,411 (google search (adsense). see press release available at https ://ec.europ a.eu/commi ssion /press corne r/detai l/en/ip_19_1770. 24 ie the positions as the default search engine on mobile devices and web browsers. 25 throughout this article some numbers are presented as ranges, and precise numbers are redacted for confidentiality reasons. another key input to producing more relevant search results is an extensive and up-to-date web-index. web-indices are repositories of data about the information contained on websites and webpages across the internet. they are used by search engines to select and return relevant search results when a consumer enters a search query. developing and maintaining a web-index is subject to cost-based economies of scale: the costs that are associated with crawling and indexing the web are substantial and do not increase proportionally with the number of users of the search engine. 26 web-indexing is also subject to cross-side network effects: the more users that a search engine has, the more value that it offers to webmasters, and the greater is the incentive that they have to make their websites accessible to that search engine. in addition, google holds extensive search default positions, including on apple's safari browser, android devices, and browsers such as mozilla firefox. google paid around £1.2 billion in default payments in 2019 in the uk alone, with the substantial majority of this being paid to apple. consistent with research in other settings, the cma found that defaults have a significant impact on consumer behaviour in search. the evidence supporting this includes google's relatively higher share of search on mobile devices (where it holds more default positions) as compared to desktop devices, 27 the limited case studies in which browsers have switched default, and internal documents in which google and microsoft modelled the search volumes that they would lose and gain respectively, if apple were to switch the safari default from google to bing. from a competition perspective, the problem is that device manufacturers and browsers generally choose defaults based on search engine quality and financial compensation (typically a search advertising revenue share). search engines other than google face barriers to competing on either of these dimensions. google's strong brand and perceived higher quality means that other search engines would likely have to offer at least as much financial compensation as google in order to win a default contract. however, google can generate more queries through a given default position than can other search engines and has superior monetisation per query due to its greater scale. as one competitor put it: 'there is a feedback loop between google's position as the largest search engine and its ability to acquire extensive default positions that entrench and reinforce this dominance'. therefore, existing rivals to google and prospective entrants face a series of selfreinforcing barriers to entry and expansion that limit the competitive threat that google faces. google's scale helps it to improve further the quality of its search results and to pay for extensive default positions. in contrast, rivals lack the scale that would enable them to improve their quality and monetisation, which in turn 27 google search has default agreements that cover much more of the mobile device sector (at least 94%) than the desktop pc sector (29%). in turn, google has a relatively higher share of supply in mobile search (97%) than it does in desktop search (84%). restricts their ability to access consumers, build their scale, and compete more effectively. the cma also examined the competition that google faces from specialised search providers: businesses that specialise in paid listings in particular sectors, e.g. booking holdings 28 in travel. a small proportion of 'commercial' search queries generate most of google's search revenues, and rival specialised search services are active in each major commercial content category. the cma found that the relationship between specialised search and google search is more vertical rather than horizontal, with google being a 'gatekeeper' for traffic to specialised search websites. while specialised search providers exercise some competitive constraint on google, through attempting to attract consumer traffic directly rather than via google's search engine, the cma's analysis of traffic data 29 shows that in most cases they were still heavily reliant on google as the main route for access: most rely on google for at least 40% of their traffic. the cma also found that these specialised search providers spent on average around 25% of their uk revenues on search advertising on google in 2019, with most spending between 15 and 30%. the cma found that google is able to exploit this dependency by employing strategies that limit the traffic to specialised search providers, making it more difficult for them to develop their services and brands and limiting the competition that google faces over the longer term. the cma heard concerns about various such strategies whereby google may leverage its market power in search to specialised search markets: first, google is able to self-preference its own specialised search services by placing links to them in 'one-boxes' that are prominently at the top of the search engine results where the user is more inclined to click. second, google may exploit the data it collects from its ecosystem to get a competitive advantage over specialised search rivals. third, google may update its organic search algorithms to demote traffic to specialised search rivals: for example, by favouring organic content directly from merchants over those of specialised search providers. these concerns are consistent with google pursuing a platform envelopment strategy in order to protect its core search advertising revenues. the cma considered the extent to which google's market power enables it to earn higher revenues than its competitors in digital advertising. ultimately, if advertising costs are higher, we would expect consumers to be harmed because they face higher prices for final goods and services. first, the cma compared the advertising prices charged by google and bing on the same search terms. the cma collected data on all the search queries that were submitted to google and bing in the uk in a single week in 2020 (several billion queries in total) and matched identical queries observed by both google and bing before comparing prices. carrying out the comparison on like-for-like overlapping queries allowed the cma to isolate differences in prices driven by market power from differences driven by the distribution of queries across google and bing. this analysis found google's prices are on average 30-40% higher than bing's across the one-week sample. it also found that google has a higher price-bid ratio 30 for like-for-like queries on average, by 10-20% on desktop and 20-30% on mobile. this suggests that google is able to extract more advertiser revenues from its auctions than can bing. this is consistent with google exploiting market power in its search auctions. it is also consistent with google benefiting from a superior product that is able to drive greater advertising returns, in part due to data or scale advantages arising from google's market power on the consumer side of the market. second, the cma examined how google's ability to monetise search has changed over time. google's search advertising revenues have been steadily increasing over the past 10 years, from £2.1bn in 2010 to £6.8bn in 2019. these revenues have grown at a significantly faster rate than the number of searches, implying that the revenue earned by google for each search has increased from a low of £0.02-0.03 per search in 2011 to a high of £0.04-0.05 per search in 2019. the cma examined the drivers of this increase in monetisation: it found that prices have been fairly stable and that the number of searches that show ads has also remained stable. however, the quantity of advertising sold (the number of clicks) for each search showing ads has increased substantially. this implies that a major driver of google's increasing monetisation over time is the 'click-through rate': the propensity for consumers to click on ads rather than organic links. google can influence the click-through rate by determining both the overall limit on the number of ads that appear in search results (the 'ad load') and how these ads are presented alongside organic search results. showing a greater proportion of ads relative to organic search results can increase the propensity of users to click on ads, which drives up the quantity of ads sold. over the past 10 years, google made several significant changes in the way that it presents ads: most notably in 2016 it removed right-hand side ads and increased from three to four the number of ads that are eligible to appear above the organic search results. a higher ad load can lead to higher costs for advertisers because it will tend to reduce the proportion of clicks that go to organic search, which websites receive for free, rather than to paid advertising. therefore, businesses that rely on accessing consumers through general search will need to pay increased advertising costs in order to maintain the same volume of overall traffic. in addition to google's ability to exploit market power by earning revenues from search advertising, the cma also considered google's ability to extend its market power into other markets. one important example relates to google's role in the open display market, where online publishers sell advertising space on their website through third party intermediaries. the cma's analysis focused primarily on three main issues: first, google's position in the adtech intermediation; second, its ability to leverage its strong position in search advertising and its wider ecosystem into the open display market; and third, the potential for google to exploit its market power and integrated position across the open display market to self-preference its own activities, thereby reinforcing its market power. publishers, such as online newspapers, rely on intermediaries to sell their advertising inventory 31 in the open display market. these intermediaries-which are sometimes known as the 'ad tech stack'-provide technologies that achieve the complicated task of selecting an ad to be served in real time and establishing the price to be paid for doing so. the main intermediaries in open display include: a. demand side platforms (dsps)-provide a platform that allows advertisers and media agencies to buy advertising inventory from many sources. b. supply side platforms (ssps)-provide the technology to automate the sale of digital inventory. they allow real-time auctions by connecting to multiple dsps, collecting bids from them, and performing the function of exchanges. c. publisher ad servers-manage publishers' inventory and are responsible for the decision logic underlying the final choice of which ad to serve, based on the bids that are received from different ssps (possibly through header bidding solutions) and the direct deals agreed between the publisher and advertisers. the cma found that google has a very strong position at all of the main stages of the adtech stack, as shown in fig. 5. google's presence across the open display market was initially driven by a series of acquisitions of publisher-facing services including doubleclick, which formed the basis for google's publisher ad server. in addition to these acquisitions, google has been able to leverage its position from its wider ecosystem into open display in several ways. google has been able to leverage the importance of youtube for advertisers to increase its market power in the dsp market by allowing advertisers to buy you-tube inventory programmatically only through google's dsp: dv360. for the many advertisers who want to include youtube inventory in their campaigns, there is a strong incentive to use dv360 for the entire campaign. google has leveraged its search advertising customer base through the convenience to advertisers of buying search and display advertising together in its other dsp: google ads. google ads is the main route through which advertisers-especially smaller ones-buy google's search inventory. by providing a one-stop shop for those advertisers that genuinely want to make use of both search and display advertising, and by nudging other small advertisers into using display ads through default settings, google has leveraged part of its search advertiser base to increase its importance as a source of demand in open display. google has also leveraged the data that are collected from its customer-facing services, including its search engine, by making these data available only to those advertisers who use its dsps. google has exclusive access to a large amount of user data that can be used for targeted advertising. data that are collected on its dominant search platform are particularly valuable for targeting purposes in open display as the data reveal users' purchasing intent. google's strong integrated competitive position gives it the incentive and ability to self-preference its own activities at each stage of the adtech stack. on the advertiser side of the market its dsps benefit from access to unique inventory, data, and its search advertising customer base. on the publisher side, there are high costs to switching ad servers, and google supplies nearly the whole market. in a future scenario where auctions are run by browsers, google would be in the position of being able to integrate the most commonly used browser-chrome-with the largest dsps. google's ability to self-preference is further supported by the lack of transparency in open display. advertisers and publishers are unable to observe easily all of the intermediaries that are involved in the buying and selling of advertising inventory along the supply chain, and there is limited visibility of fees or bidding data in advertising auctions. this makes it more difficult for advertisers and publishers to understand or monitor potential conflicts of interest. the publisher ad server is particularly important as it is responsible for the decision logic that underlies the final choice of which ad to serve. this enables it to set or influence aspects of the auction processes in open display that other intermediaries then need to abide by. google has used its position as the largest publisher ad server to influence auction processes in various ways that favour its own dsps and ssp (adx), though its approach has evolved rapidly along with the evolution of the technologies that are used in open display. google has also made it difficult to access its advertiser demand through alternative publisher ad servers, thereby increasing its market power in ad serving and making it difficult for other providers to compete on the merits. one aspect of this is that google channels much of the demand from its dsps to its own ssps. the other is that publishers have a strong incentive to use google's ad server to ensure that there is real-time competition between adx and rival ssps, because adx does not participate in header bidding. 32 the cma found evidence that intermediaries-including google-capture at least 35% of the value of advertisers' purchases across the open display market as a whole, which is consistent with the findings from other studies on ad tech 'take rates', 33 including a recent isba/pwc study, 34 the plum report, 35 and the ana analysis. 36 google's position and behaviour in open display is a source of concern for three reasons: first, although we did not find evidence that google is currently charging higher fees than its competitors, google would have the ability to increase fee levels in the future once its dominance is consolidated. second, the reduction in competition in parts of the ad tech stack could also have dynamic effects, with a reduction in pressure to innovate and create new products-which could ultimately harm advertisers and publishers. third, there is also the possibility that part of google's reason for establishing a strong position in open display is to protect its position in other markets. for example, entry into open display could be a way to protect google's search advertising business from potential competition if google envisages a trend towards greater convergence of search and display advertising. the cma found that facebook has significant and enduring market power in social media and display advertising. on the user side of its platforms, facebook (including instagram and whatsapp) has a reach of over 85% of uk internet users and around 75% of the time spent on social media for a number of years. this finding is consistent with the results of other investigations, such as the accc digital platforms inquiry. 37 network effects are a key source of facebook's market power. social media platforms-particularly those that focus on communication-become more valuable to users as other people that they want to interact with join the platform. facebook's very large network of connected users means that people can use it to network with close contacts (the most important reason why consumers access facebook) and more distant connections alike. cross-side network effects further reinforce facebook's position and act as a barrier to expansion for smaller platforms. in addition to social networking, facebook serves content-related needs such as video streaming, and other needs such as gaming and shopping. cross-side network effects have supported this in several ways: first, facebook's large network of connected users attracts app developers and content providers, which further increases its value to consumers. second, facebook's many users have increased its value to advertisers, providing it with high revenues and greater scope to invest in its platform and broaden its services. entrants have responded by differentiating and building different types of services, rather than social networks that compete more directly with facebook. for example, snapchat is primarily used to communicate with close friends and family members, and linkedin offers professional networking. this differentiation limits the degree of substitutability between social media platforms for consumers and means that the constraint placed on facebook is limited. there has been no successful entry in the last 10 years by a direct competitor with a comparable set of services to those provided by facebook, with google's attempt (google +) having failed. although many consumers use multiple social media platforms, they generally use third-party platforms alongside-rather than as an alternative to-facebook. for example, as is seen in fig. 6 , 80% or more of the audiences of tiktok, snap-chat, and twitter also use facebook. conversely, a much smaller proportion of facebook's audience also visits these third-party platforms. 38 facebook is also able to affect the competitive conditions it faces. interoperability between platforms has the potential to reduce the extent to which network effects act as a barrier to entry and expansion. however, facebook provides only limited 37 accc (2019). 38 this result also holds for the other third-party platforms that we analysed, including linkedin, reddit, pinterest and tumblr. interoperability with other platforms and in some cases has changed or reduced the interoperability that it offers. for example, the deprecation of two functionalities appears to have harmed the ability of other platforms to grow: a. "find friends"-historically, third-party developers could enable their users to invite their facebook friends (including friends not using the relevant app) to use an app through tools such as app invites. the removal of this functionality in 2018 meant that the list of facebook friends that can be invited is limited to those friends who have also signed up to that app: 'in-app friends'. b. "publish actions"-historically, this enabled third-party applications to publish posts to facebook as the logged in user so that users could easily share content on facebook that they have created elsewhere. this functionality was deprecated in 2018. facebook's market power is likely to have several negative impacts for consumers. first, it means that facebook has weaker incentives to innovate and to develop its platforms in ways that are valued by consumers. second, facebook can extract more consumer data or worsen the terms that it offers consumers for these data. third, consumers are harmed indirectly through higher prices for other goods and services if facebook raises the prices it charges for display advertising above competitive levels. facebook's market power on the consumer side of its social media platform has allowed it to gain a large share of consumer attention that can be monetised through digital advertising. it also has a significant data advantage over smaller platforms and publishers, which both increases the value of its advertising inventory and creates additional barriers for its competitors to overcome. this has led to facebook also benefiting from market power in display advertising and has allowed it to earn substantially higher revenues per user than its competitors, increasing from an average of £0-5 in 2011 to £50-60 in 2019 (see fig. 7 ). facebook (including instagram) generated over half of uk display advertising revenues in 2019. another way in which facebook is able to take advantage of its market position in social media is through leveraging its market power into related markets: for example, through its marketplace and gaming services and its portal devices. the cma heard a range of concerns about potentially exclusionary practices. for example, there were concerns that facebook is able to bundle new services into its pre-existing social media platform and that it can unfairly obtain access to data on its rivals' customers when it provides these rivals with advertising services or the developer tools that are needed to interoperate with facebook's social media platform. the cma found that facebook may have the ability and incentive to engage in exclusionary practices in related markets. facebook's ability stems from its position in social media, which results in its being a critical partner for businesses in related markets, including those that rely on its 'free' developer tools and/or its advertising services. facebook may also have the incentive to engage in exclusionary practices to protect its position in social media, or to gain additional profits from the most lucrative related markets by offering these products itself. the cma found that both google and facebook have entrenched market power, that has arisen primarily from network effects and other scale advantages from the consumer sides of their platforms. they are able to exploit this market power by monetising the consumer attention and data through digital advertising for which they can charge high prices. both google and facebook have behaved in ways that entrench their market power or leverage it when expanding into other related markets. in the long run, this may have a substantial negative impact on incentives for innovation and investment. the cma concluded that google and facebook have such entrenched market power that the cma's current tools are not sufficient to protect competition. tackling such issues requires an ongoing focus, and the ability to monitor and amend interventions as required. in addition, the markets in which google and facebook operate are fast-moving, and the issues within them are wide-ranging, complex, and rapidly evolving. therefore, the cma made recommendations to government to develop a procompetition ex ante regulatory regime for online platforms. this regime would be comprised of two elements: an enforceable code of conduct, and a range of procompetition interventions. the function of the enforceable code of conduct would be to govern the behaviour of platforms that enjoy a position of market power. it would apply to platforms with 'strategic market status' (sms), as envisaged by the furman review. the objective of the code would be to address the harmful exercise of market power. the cma proposed that the code of conduct would take the form of highlevel principles rather than detailed and prescriptive rules and would be organised around three high-level objectives: fair trading principles are intended to address concerns about the potential for exploitative behaviour on the part of the sms platform. open choices principles are intended to address the potential for exclusionary behaviour. trust and transparency principles are designed to ensure that the sms platform provides sufficient information to users, so that they are able to make informed decisions. pro-competitive interventions, in contrast, would be designed to tackle the sources of market power and promote competition and innovation. the cma proposed that powers to introduce the following three forms of intervention were necessary: a. data-related interventions, such as increasing consumer control over data, mandating third-party access to data, mandating interoperability, or mandating data separation. these proposed interventions reflect the fact that differential access to data is at the heart of many important barriers to entry and expansion. b. consumer choice and default interventions, which restrict platforms' ability to secure default positions, such as google has done in securing default positions for its search engine on browsers and devices. c. separation interventions, which address the potential for vertically integrated platforms to self-preference their own activities. on 13 february 2020, the cma cleared google's $2.6bn acquisition of looker. 39 this was one of the largest acquisitions by a major tech firm since the publication of the furman review. 40 that review, and a number of other studies, 41 had expressed concerns about underenforcement in mergers involving digital firms and the need for competition authorities carefully to consider a comprehensive range of possible competition concerns. 42 this case was an early opportunity for the cma to address those concerns. the acquisition of looker added to google's cloud computing business. google is currently the third largest cloud computing provider behind amazon and microsoft. one of google's cloud computing products is its data warehouse: google bigquery. data warehouses are databases that are designed to enable analytics, and they are often used to combine data from multiple sources. possible sources of these data include data from google's advertising and web analytics products. firms commonly analyse the data that are stored in a data warehouse with the use of a business intelligence (bi) tool. bi tools are used to analyse, visualise, and interpret data to support corporate decision-making. looker is a provider of bi tools. 43 the rationale for the merger was to improve google's cloud computing offering, which would allow it to compete more effectively with the leading cloud computing providers: in particular, amazon and microsoft. the cma's focus was on the potential for google to foreclose competing bi tools by restricting access to data that are: (i) stored in google bigquery; and (ii) generated by google's advertising and web analytics products. this assessment illustrates: a. the adaptability of the uk's mergers regime to consider a wide range of theories of harm-in this case a non-price foreclosure strategy that involves data; 39 cma (2020). 40 hm treasury (2019) . 41 stigler report (2019) and lear (2019). 42 for example, hm treasury (2019, p. 6) . 43 for example, bi tools can simplify the process of data analysis and can be used to ensure that everyone in an organisation analyses data on a consistent basis (e.g., by using common definitions for key metrics). b. the value of seeking to understand the acquirer's business model and strategy so as to understand the role of the transaction; c. the value of an inter-disciplinary approach and taking advantage of wider institutional knowledge when assessing such cases; d. the role that considering a wider ecosystem of products can play in assessing market power and when assessing a foreclosure strategy; and e. the wide range of evidence that the cma draws on in its assessments. the case was a 'phase 1' merger assessment: in these cases the cma engages in an initial period of evidence gathering and focusses on information requests to the merging firms. once the cma is satisfied with the information that has been provided by the firms, the formal investigation is launched. the cma then has 40 working days to complete its assessment. it is primarily during this period that the cma collects evidence from third parties and provides the merging firms with an opportunity to respond to any preliminary concerns. at the end of the 40-working-day period the cma must conclude (amongst other things) whether there is a realistic prospect of a substantial lessening of competition. if this test is met, the case moves to a more in-depth phase 2 assessment unless satisfactory remedies are offered. in this case, the cma initially sought to understand the set of products that are provided by google and looker. in doing so, the cma also drew on the expertise of its data, technology and analytics team, who provided important specialist input, and on the findings from the cma's digital advertising market study. the large number of products involved and the various relationships between them meant that there was a range of possible theories of harm. the cma's focus was on identifying linkages between looker and areas where google has market power. by the end of the initial period of evidence gathering the cma identified the potential foreclosure of competing bi tools with the use of google's web analytics and online advertising products as the main area for concern. this theory of harm focussed on the possibility of non-price foreclosure where access to data from these google products was the mechanism used to foreclose. specifically: a. google provides a number of advertising and web analytics products that generate data which is then analysed using bi tools such as looker; b. google also provides a data warehouse, google bigquery, that can be used to aggregate data for analysis using a bi tool; c. therefore, google could use its services that generate data and google bigquery to foreclose competing bi tool providers: for example, by removing or impeding the ability of competing bi tools to connect to the data sources that google controls. the cma assesses foreclosure theories of harm using the standard ability, incentive, effect framework. in its assessment of google's ability to foreclose competing bi tools the cma considered: a. the extent to which accessing data from google's web analytics and online advertising product is important for users of bi tools; b. whether google has market power in relation to web analytics and online advertising services; and c. the mechanisms that google could use to achieve a foreclosure strategy. to assess these questions the cma drew on a significant number of google and looker internal documents and evidence from google and looker's customers and their competitors. another important source of evidence-particularly when assessing google's market power-was the cma's online platforms and digital advertising market study that was discussed above. 44 the study published its interim report during the course of the merger inquiry, which highlighted google's strong position in online advertising services. google's web analytics product was also used by approximately 80% of uk web domains-vastly more than any other web analytics product. in its assessment of google's market power, the cma also placed significant weight on google's ability to offer a set of inter-related products. not only did each of google's advertising and web analytics products have a strong market position in isolation, but google's ability to offer a set of inter-related products reinforced this market position. for example, advertisers value data on the performance of their advertising campaign when deciding where to advertise and which products to use. google's web analytics product allows it to offer richer data to advertisers than would otherwise be the case. this finding was supported by evidence from google's internal documents. the cma's approach here illustrates the important role that a firm's 'ecosystem' of products can play in the assessment of the effects of a merger. input from the cma's data, technology and analysis team was important in assessing the mechanisms that could be used to foreclose competing bi tools. this assessment was necessarily technical and required an understanding of the means by which bi tools access and analyse data and google's ability to control these. the cma ultimately identified three possible foreclosure mechanisms (illustrated in fig. 8 ): a. some bi tools access google data sources directly using google's application programming interface (api) (route a in fig. 8 ). google can control the func-44 cma (2020). tionality of this api, and therefore the ability of competing bi tools to access these data, in a variety of ways. b. many customers will move data from google data sources to a data warehouse, a bi tool will then be used to analyse the data in the data warehouse (route b in fig. 8 ). data are moved to a data warehouse using the same api as above, and therefore, the same foreclosure mechanisms could be used. c. finally, a customer may move the data to google's data warehouse (google bigquery) before analysing it with a competing bi tool (route c in fig. 8 ). in this case, google could control the google bigquery api to hamper the ability of competing bi tools to analyse the data. on the basis of this evidence the cma concluded that google would have the ability to use a range of non-price foreclosure mechanisms to hamper competing bi tools from accessing data from google's advertising and web analytics products, and from google bigquery. 45 4.2.1.2 assessing google's incentive to foreclose a common starting point for assessing incentives to foreclose is the vertical arithmetic framework. this framework considers the extent to which customers may switch away from 46 the merging parties' products in response to a foreclosure strategy and the profit margins on those products to identify the costs and benefits of a foreclosure strategy. 47 in this case, a foreclosure strategy had the potential to increase google's profits due to: a. switching to using looker leading to an increase in sales and possibly higher prices for looker; and the cma also found that google had the ability to engage in price foreclosure. 46 switching away could involve reduced usage as well as total switching. 47 see riordan and salop (1995). b. switching to google bigquery, if the foreclosure strategy involved hampering the ability of users to transfer data from google's products to competing data warehouses (routes a and b in fig. 8 ). on the other hand, google might suffer from a reduction in profits due to: a. switching away from or reduced usage of the google products that generate the data; and b. switching away from google bigquery, if foreclosure involved hampering the ability of competing bi tools to analyse data that are contained in google bigquery (route c in fig. 8 ). as well as providing a framework for identifying the costs and benefits of any foreclosure strategy, the vertical arithmetic framework can also be applied to assess whether there is in fact an incentive to foreclose. specifically, profit margins can be used to calculate the relative degree of switching to google's products which would be necessary for foreclosure to be profitable. this can then be compared to estimates of the actual degree of switching that would be likely to occur. 48 the cma sought evidence of product specific profit margins from google and looker. however, the global and highly integrated nature of the businesses meant that it was not possible meaningfully to identify profit margins for the products of interest. this was particularly so because some products have a wider contribution than would be indicated by looking at their margins alone. for example, google provides a free version of its web analytics software in part because of this software's contribution to its wider advertising business. instead, the cma considered a range of alternative evidence including: a. internal documents and the merger rationale to assess google's view of the profitability of different strategies; b. evidence on the relative ease of switching to and from different products; c. evidence on the ability to target any foreclosure strategy at competing bi tools; and d. google's pre-merger behaviour and an assessment of the extent to which this would be affected by the merger. evidence on the ability to target any foreclosure strategy was of particular importance. a wide range of products might seek to access data that are generated by google's products or might seek to access google bigquery. many of these products will not be in competition with looker and will be complementary to google's other products. consequently, a foreclosure strategy that would be targeted at competing bi tools that analyse data only from google products would involve substantially less loss of revenue than an untargeted strategy that affected all thirdparty products. the cma considered that a foreclosure strategy could not be targeted using google bigquery (route c in fig. 8 ). in large part this was because google has no ability to ascertain the source of the data that is being analysed via google bigquery. the cma considered that as a result google would not use route c to attempt to foreclose competing bi tools, particularly because such a strategy would risk losing google bigquery customers. it would also be inconsistent with the merger rationale of enabling google to compete more effectively with the leading cloud computing providers. in reaching this conclusion, the input of the cma's data, technology, and analytics team was once again invaluable in properly assessing the ability of google to target foreclosure. the above finding had important implications for the assessment since, in theory, google already had some incentive to use the other foreclosure mechanisms (routes a and b in fig. 8 ) to favour google bigquery prior to the merger. however, there was no evidence that google had pursued this strategy pre-merger. furthermore, pursuing a more limited foreclosure strategy (using only routes a and b) would lessen the extent to which customers would be steered towards using looker. as a result, the benefits of any foreclosure strategy would be significantly reduced, as would the effect of the merger on google's incentives. ultimately, the cma considered that google was unlikely to have an incentive to foreclose competing bi tools as a result of the merger. since the cma found that google would not have an incentive to foreclose, it was not necessary to consider the effect of any foreclosure strategy. the google/looker merger investigation illustrates the adaptability of the cma's merger assessment framework to consider a diverse range of theories of harm-in this case the possibility of partial foreclosure using access to data. the cma was able to consider comprehensively a complex theory of harm within its phase 1 merger process and thereby demonstrated the uk merger regime's ability to address the concerns that have been raised in the furman review and in other reports. the assessment shows the importance of specialist knowledge and advice when analysing such theories of harm. in this case advice from the cma's data, technology, and analytics team was invaluable in evaluating the theory of harm. finally, an important aspect of the cma's analysis was a thorough understanding of the merging firms' business models and of the merger rationale. the three cases that have been presented in this article demonstrate the variety and significance of the work that was undertaken by the cma over the last year. its economists have been absolutely central to that work, and the cases that have been digital platforms inquiry programmatic: seeing through the financial fog price gouging. retrieved 13 completed acquisition by google llc of looker data sciences, inc. decision on relevant merger situation and substantial lessening of competition harnessing platform envelopment in the digital world excessive prices: using economics to define administrable legal rules are excessive prices really self-correcting? what should we do about price gouging? economics observatory tackling the covid-19 challenge-a perspective from the cma unlocking digital competition: report of the digital competition expert panel. london: the stationary office programmatic supply chain transparency study excessive pricing in competition law: never say never. the pros and cons of high prices the ethical content of the economic analysis of disasters: price gouging and post-disaster recovery evaluating vertical mergers: a post-chicago approach draft report by committee for the study of digital platforms, market structure and antitrust subcommittee acknowledgements we thank julie bon, francesca botti and simon roberts for helpful input. the views that are expressed in this article are those of the authors and do not necessarily reflect those of the cma. discussed in this article reflect the value of economic input in guiding appropriate interventions.publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. 