Leveraging Informatics and Technology to Support Public Health Response: Framework and Illustrations using COVID-19 1 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 13(1):e1, 2021 OJPHI Leveraging Informatics and Technology to Support Public Health Response: Framework and Illustrations using COVID-19 Jane L. Snowdon PhD1*, William Kassler MD MPH1, Hema Karunakaram MPH1, Brian E. Dixon MPA PhD2,3, Kyu Rhee MD MPP1 1IBM Watson Health, Cambridge, MA, USA 2Center for Biomedical Informatics, Regenstrief Institute, Indianapolis, IN, USA 3Department of Epidemiology, Richard M. Fairbanks School of Public Health, Indiana University, Indianapolis, IN, USA Abstract Objective: To develop a conceptual model and novel, comprehensive framework that encompass the myriad ways informatics and technology can support public health response to a pandemic. Method: The conceptual model and framework categorize informatics solutions that could be used by stakeholders (e.g., government, academic institutions, healthcare providers and payers, life science companies, employers, citizens) to address public health challenges across the prepare, respond, and recover phases of a pandemic, building on existing models for public health operations and response. Results: Mapping existing solutions, technology assets, and ideas to the framework helped identify public health informatics solution requirements and gaps in responding to COVID-19 in areas such as applied science, epidemiology, communications, and business continuity. Two examples of technologies used in COVID-19 illustrate novel applications of informatics encompassed by the framework. First, we examine a hub from The Weather Channel, which provides COVID-19 data via interactive maps, trend graphs, and details on case data to individuals and businesses. Second, we examine IBM Watson Assistant for Citizens, an AI-powered virtual agent implemented by healthcare providers and payers, government agencies, and employers to provide information about COVID-19 via digital and telephone-based interaction. Discussion: Early results from these novel informatics solutions have been positive, showing high levels of engagement and added value across stakeholders. Conclusion: The framework supports development, application, and evaluation of informatics approaches and technologies in support of public health preparedness, response, and recovery during a pandemic. Effective solutions are critical to success in recovery from COVID-19 and future pandemics. Keywords: coronavirus, pandemics, public health informatics, clinical informatics, artificial intelligence, information technology Correspondence: snowdonj@us.ibm.com* DOI: 10.5210/ojphi.v13i1.11072 mailto:snowdonj@us.ibm.com* Leveraging Informatics and Technology to Support Public Health Response: Framework and Illustrations using COVID-19 2 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 13(1):e1, 2021 OJPHI Introduction The United States has faced numerous pandemics including the HIV/AIDS pandemic; the Zika, MERS, and Ebola outbreaks; and the current novel coronavirus disease 2019 (COVID-19) pandemic. Each experience has revealed opportunities for improvement by highlighting unmet data and information needs [1] among clinicians, public health agencies, policymakers, and researchers during the three stages of a disease outbreak: preparedness, response, and recovery. Some of these prior lessons are summarized in Table 1, adapted from an article by Paules et al. [2]. Table 1. Optimal Response to Emerging Infectious Disease Outbreaks: Lessons Learned Global Surveillance to detect outbreaks easily Transparency and communication in response to outbreaks Incorporation of infrastructure and capacity building domestically and internationally in outbreak responses Conduct of basic and clinical research associated with outbreaks in a coordinated and collaborative manner Involvement of the afflicted communities in policy decisions Pursuit and perfection of adaptable platform technologies for vaccines, diagnostics, and therapeutics Importance of flexible funding mechanisms Benefits of data standards and standardization to facilitate the interoperability of solutions Critical need for investing in skills, training, and skills in science, technology, engineering, and mathematics (STEM) to grow national competencies In just 6 months, COVID-19 spread rapidly from the first reported cases of the SARS-CoV-2 virus in Wuhan, China in December 2019 across the globe causing 9,843,073 cases and 495,760 deaths in over 185 countries as of June 28, 2020 [3,4]. Unlike prior pandemics, COVID-19 response and recovery will rely more heavily on informatics and technology given their extensive diffusion into business, health care and everyday life. Already the US has leveraged cloud computing, video conferencing, collaboration tools, and digital security solutions to support remote working by employees, as well as artificial intelligence (AI) to diagnose COVID symptoms [1,5-7]. Copyright ©2021 the author(s) This is an Open Access article. Authors own copyright of their articles appearing in the Online Journal of Public Health Informatics. Readers may copy articles without permission of the copyright owner(s), as long as the author and OJPHI are acknowledged in the copy and the copy is used for educational, not-for-profit purposes. Leveraging Informatics and Technology to Support Public Health Response: Framework and Illustrations using COVID-19 3 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 13(1):e1, 2021 OJPHI Prior lessons and frameworks for public health response do not focus on the roles of informatics or technology, and they tend to be heavily focused on vaccine and therapeutic development [2]. The informatics community needs a method for framing COVID-19 response and lessons learned to highlight the diverse ways that computing, algorithms, and informaticians contribute to addressing the pandemic. Such a framework could be used by researchers, policymakers, and organizational leaders to measure response activities to COVID-19, monitor recovery, and help to prepare and plan for future health threats. The objective of this paper is to describe and illustrate a novel, comprehensive framework that encompasses the myriad ways informatics and technology can support public health response to a pandemic like COVID-19. In this context, public health response spans government, industry, academia, and citizens, and is not solely limited to government. The pandemic framework supports and expands the broader concept of Public Health 3.0 [8,9]. Two case examples showing early outcomes of how informatics and technology are supporting the U.S. response to COVID-19 are given. Consideration of health informatics and technology from an outbreak’s local onset to a global pandemic relies on trusted sources of public health protocols, plans, and data from governmental organizations such as the World Health Organization (WHO), Centers for Disease Control and Prevention (CDC), Centers for Medicare and Medicaid Services (CMS), and from the private sector such as The Weather Company (TWC). Having good data is critical for developing predictions, assessing risks, allocating resources, and measuring the effectiveness of interventions, such as wearing masks and social distancing. Incorrect or incomplete data may cause harm by omitting important socioeconomic factors, obscuring trends or correlations, and may lead to ill- informed or incorrect actions by decision-makers. Methods The COVID-19 pandemic highlights the complexity and challenges in responding to a significant, widespread man-made or natural disaster including emerging infectious diseases. Saving lives, maintaining critical infrastructure, and restoring essential services require coordinated response across various governmental and private sector organizations to mobilize resources and capabilities to provide support and services [10]. These tasks involve a diverse set of functions that are data-rich and heavily quantitative. Informatics and technology can play significant supportive roles before, during, and after the public health emergency. Figure 1 illustrates a conceptual model of public health preparedness for major health threats. The model is adapted from the traditional public health stages of disease outbreak [11] and expanded to incorporate stakeholders and sectors beyond governmental public health. Note that there is overlap among the set of stakeholders who are not mutually exclusive. Leveraging Informatics and Technology to Support Public Health Response: Framework and Illustrations using COVID-19 4 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 13(1):e1, 2021 OJPHI Figure 1. A conceptual model of public health preparedness encompassing the phases, economic sectors, and cross-sector organizations that engage in preparing, responding, and recovering to major health threats. Prepare Preparing for a crisis requires an understanding of the potential challenge, having the foundation in place to assure continuity of operations, developing the plans and policies to enable a rapid response, and building the infrastructure to support those in need. Taking steps to reduce vulnerability to disaster before the emergency is critical for success. Unlike some natural disasters that occur regularly and with predictable periodicity (e.g., hurricanes, fires), pandemics are spontaneous and unpredictable. Because infectious disease outbreaks can rapidly spread across the globe, having plans, policies and resources in place allows government officials to take early steps to limit disease from spreading. While it is impossible to plan for every contingency, building capacity to respond can significantly mitigate the impact on a community. Respond Responding is the period when stakeholders implement plans, translate science into practice, and course-correct based on new data to improve practices. As exhibited during COVID-19, the impact of a rapid and widespread, novel infectious disease can cause massive disruption across many sectors of the economy. Thus, pandemic response is a coordinated effort across multiple sectors. Leveraging Informatics and Technology to Support Public Health Response: Framework and Illustrations using COVID-19 5 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 13(1):e1, 2021 OJPHI In the area of basic science, response is mobilizing to characterize the etiologic agent and its genetics, and to develop diagnostic tests, drugs and vaccines. The life sciences industry plays a critical role in executing the applied science and manufacturing needed to develop vaccines, tests and treatments, and to successfully bring them to market. In the healthcare sector, payers, health plans and providers work in concert to assure those in need get services in the face of an unprecedented surge in demand resulting from local outbreaks. Government has a unique role in helping to monitor the situation, coordinate across numerous public and private organizations, mobilize supportive assets and provide financial resources. Epidemiology plays a key role in understanding patterns of transmission, describing the clinical course, individual and population risk factors, quantifying the burden of illness, and creating mathematical models to predict demographic and geographic spread of disease. Public health departments focus on interventions to mitigate the spread, including case finding, contact tracing, quarantine, and isolation. Communication and dissemination of information is critical to response, particularly in the face of rapidly evolving events. Government is the natural leader for overall coordination and communication efforts. Government also plays a key role in translation of newly emerging evidence into best practices and policy. Disasters often stress the economy and, as in the case of COVID-19, expose structural weaknesses revealing significant gaps in organizational capacity, staff and resources. Private industries across economic sectors work to assure continuity of operations, repairing broken supply chains and creating and adapting to new ways of working to mitigate transmission. Recover Pandemic recovery takes time. Even before the pandemic recedes, many recovery actions can begin, starting with measured re-opening of business, schools and other public spaces, as well as providing short-term relief for people most impacted. Intermediate-term recovery involves dealing with the secondary impact. As the epidemic curve wanes and healthcare surges decline, additional impact from secondary surges will be felt. New curves will arise resulting from the increased demand following disruptions in routine healthcare. Delayed primary and secondary prevention will result in surges of pent up demand for elective surgical procedures, in routine care for chronic disease, and in increased admissions for ambulatory sensitive and prevention sensitive conditions (e.g. increased infectious disease outbreaks for vaccine preventable diseases, increased admissions for poorly controlled diabetes and asthma, and more presentation of severe and later-stage illness such as cancer). Given the stress, anxiety, social isolation, economic hardships, and collective trauma, another surge in demand for mental health services is also expected. Long-term recovery means building individual and community resiliency. This involves an after- action review of the responses carried out during the crisis, a formal evaluation of what interventions did and did not work in order to understand best practices and lessons learned, and feedback of insights into the preparation stage to improve future emergency responses. Leveraging Informatics and Technology to Support Public Health Response: Framework and Illustrations using COVID-19 6 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 13(1):e1, 2021 OJPHI A Framework for Pandemic Preparedness, Response, and Recovery Building upon this conceptual model, we developed a framework for characterizing informatics and technology support of the diverse tasks involved in public health preparedness, response and recovery as depicted in Figure 2. Each category consists of multiple informatics and technology solutions that can be leveraged during one or more phases of a significant health threat. Furthermore, each category is primarily driven by a specific stakeholder group. It should be noted that multiple stakeholders may play a role in funding, implementing, or using the informatics and technology solutions that correspond to a given category. Figure 2. A framework for leveraging informatics and technology to support public health preparedness, response, and recovery. Benchside Benchside solutions focus on applied science and span the “prepare” and “respond” phases of the pandemic. Life sciences companies who have invested in informatics solutions for drug and vaccine development can quickly ramp up to address a pandemic. Automated digital tools, including AI, can support drug development with clinical trial study protocol design and development, and electronic data capture and management, to shorten the timeline to release new drugs and vaccines to the market [12-15]. Bedside Bedside solutions support clinical care and decision-making for provider organizations in the “respond” phase. These solutions may help automate triage of patients seeking care based on symptoms or may reallocate clinical staff across an organization based on demand for services. Leveraging Informatics and Technology to Support Public Health Response: Framework and Illustrations using COVID-19 7 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 13(1):e1, 2021 OJPHI Reeves et al. [16] describe the design and rapid creation of a COVID-19 operational dashboard by a multidisciplinary team of providers and administrators. The dashboard provided real-time data and analytics on the number of patients tested, test results, intensive care unit bed availability, ventilator unit availability, and ambulatory visit volume. Disease Investigation, Modeling and Monitoring Epidemiologists use a variety of tools that span the “respond” and “recover” phases of the pandemic to model disease outbreak and monitor infectious disease cases. Surveillance helps provide situational awareness and enhances early detection and response. Public and private scientific organizations may use these tools to map the spread of disease, track infection rates and outcomes across populations, and predict where more resources need to be allocated. As the understanding of the disease evolves, solutions can also assist contact tracing, quarantine and isolation efforts to contain the disease. One example is Spatiotemporal Epidemiologic Modeler (STEM), an open source software project supported by the Eclipse Foundation, which is used by a global community of researchers and public health officials for disease modeling and tracking [17]. Another example is the Regenstrief COVID-19 Dashboard, deployed to support public health and health systems monitor the evolving situation in the State of Indiana [18]. Community Coordination and Programs Once a pandemic is declared, public health agencies activate a number of “respond” and “recover” activities. These efforts seek to mobilize community-based organizations and public health programs to help individuals, families, and at-risk populations. Examples include establishing additional screening or testing locations to identify impacted populations, targeted testing efforts in specific locations (e.g., nursing homes, food processing plants), or erecting field hospitals that might serve as triage sites or post-acute recovery centers. Public health agencies also initiate programs that address social needs such as housing for individuals without a safe place to isolate, or feeding individuals who are food insecure during quarantine. These programs are typically coordinated with governmental social services agencies and community-based organizations that may manage the programs or contribute volunteers to ensure sufficient operation. Communications & Automated Assistance Communications and automated assistance solutions are critical across all phases of a pandemic but are primarily used during the “respond” phase. Government agencies, providers, and payers may all experience an influx of questions from individuals wanting to understand new policies, check symptoms, or find assistance. Solutions such as virtual assistants, conversational agents and chatbots may be employed to broadly communicate changes in local guidelines, automatically answer common questions about the pandemic, and triage more complex questions to the appropriate sources of information. Return to Work As the pandemic wanes and social distancing measures gradually lift, focus shifts to restoring businesses and services. Businesses that have been impacted by the pandemic will be eager to Leveraging Informatics and Technology to Support Public Health Response: Framework and Illustrations using COVID-19 8 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 13(1):e1, 2021 OJPHI open, yet returning to the workplace before a therapeutic drug or vaccine are available poses risks to front-line employees and organizations. Employers need to plan for a safe way to resume operations. These solutions may guide the transition of employees back into work environments and inform critical business processes to ameliorate negative economic impacts from the pandemic. Decision-makers are faced with a sense of urgency: when to reopen and how to do so in a responsible manner. Such decisions require analyzing and balancing numerous factors to support tailoring evidence-based return to workplace policies for different sites and job roles. Factors include monitoring local infection rates and population health trends; gathering and monitoring employee symptoms, assessing their status and referring employees to appropriate testing and medical care; weighing employee vulnerability; understanding state and local regulations; and responding to a range of employee questions in a scalable and comprehensive way. Business Continuity and Resiliency Business continuity and resiliency solutions are focused on anticipating and addressing disruptions to business operations and span the “prepare” and “respond” phases of the pandemic. Business continuity solutions may help organizations more easily transition to remote working, keep essential systems running for users, and bolster cloud environments. Supply Chain Disruption To ensure smooth operations across an organization’s supply chain, solutions addressing potential disruptions to supply chains span all three phases of the pandemic response. Proactive investment in these solutions can ensure that key materials and parts have backup suppliers and customers can continue to access essential products. New Ways of Working Solutions that enable new ways of working look to the future across communication and business continuity and span the “respond” and “recover” phases of the pandemic. Employers may invest in solutions that enhance productivity, collaboration, and learning opportunities for employees who are working from home, learning from home, or transitioning to new business processes. Mapping We mapped IBM’s existing commercial solutions and technology assets to the pandemic framework, which identified some gaps in capabilities and opportunities for building new products where we could make the largest impact based on our resources and the needs of our stakeholders. The company responded quickly, embraced a change in culture by breaking down silos, and united behind common goals to either integrate and bundle existing products and services to address urgent customer requirements or to develop new solutions with speed and agility. A few of the new solutions that were developed based on this mapping are described in the results section. Leveraging Informatics and Technology to Support Public Health Response: Framework and Illustrations using COVID-19 9 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 13(1):e1, 2021 OJPHI Results To illustrate the framework, we present two case examples that detail how informatics and technology are supporting the U.S. response to COVID-19. These examples represent early findings from technologies actively in use, which may evolve further as the pandemic continues. Both examples are from the Communications and Automated Assistance category in Figure 2. The Weather Channel COVID-19 Information Hub The Weather Channel app and weather.com website provide weather forecasts, current conditions, interactive radar, satellite maps, real-time severe weather alerts, and offer tools that predict local risk of flu or seasonal allergy symptoms up to 15 days in advance at a city level [19,20]. Combined, the website and app serve more than 300 million monthly users globally. As COVID-19 spread, The Weather Channel team developed and deployed in less than 1 month a COVID-19 Information Hub (see weather.com/coronavirus) [21]. The Hub provides interactive maps, data on confirmed cases, trend graphs, a Q&A chatbot, and news about COVID-19. The Hub, which runs on a hybrid cloud infrastructure, uses Watson AI to integrate relevant COVID- 19 data from the WHO, CDC, and many state as well as local governments every 30 minutes. Where available, data is provided at the county level in the United States [22]. For those seeking additional data, IBM created a separate interactive dashboard, built using IBM Cognos Analytics on IBM’s public cloud, available at https://accelerator.weather.com. The dashboard provides users, such as researchers, data scientists, and media, with analysis and filtering of regional COVID-19 data. Users can drill down to the region, country, state, and even county and leverage different visualizations (charts, maps, graphs) to examine case as well as mortality trends. Since launching on March 26, The Weather Channel COVID-19 Hub has seen an average of 2.9 million visitors daily and more than 299 million visits (as of June 28, 2020). The ‘COVID-19 Q&A with Watson’ chatbot has answered more than 2.2 million questions, with “signs and symptoms” being the most popular topic. In a survey of users in April 2020 to gather reaction to the COVID- 19 hub, participants (N=5,422) expressed a high degree of satisfaction (mean 4.38 using a 5-point Likert scale). The most desired information focused on areas where cases are concentrated (63%), severity of cases in their local area (62%), predictions for cases in their area (60%), predictions for when the COVID-19 outbreak would end (58%), and for confirmed cases (54%). Watson Assistant for Citizens Healthcare providers and payers, government agencies, and businesses faced staggering increases in telephone call volume as a result of the COVID-19 pandemic. Patients, employees, and citizens have questions requiring timely and accurate responses about COVID-19 symptoms and case counts, how and where to get tested, what to do if they recently traveled, whether schools and facilities are closed, what a facility’s emergency plan is, whether events are cancelled or postponed, and more. The rapid pace of change in the evolution of the pandemic is further challenging communications. From tracking the disease and local response, to keeping up with the Leveraging Informatics and Technology to Support Public Health Response: Framework and Illustrations using COVID-19 10 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 13(1):e1, 2021 OJPHI emerging clinical and public health science, organizations must ensure that their communication is accurate, timely and up-to-date. Vetting and curating that content is no small task. Institutions are establishing call centers and deploying virtual agents to assist in providing accurate, reliable and timely answers to common questions related to COVID-19. Watson Assistant, a conversational AI platform that supports both multi-channel text and speech-based interfaces, combined with natural language processing technology and Watson Discovery, an enterprise AI search technology for unstructured data, enables virtual agents to dynamically crawl the CDC website daily for the most current COVID-19 guidance as well as state websites and local sources about school closings and news. In Table 2 we highlight three institutions that used Watson Assistant to manage inquiries from the public when staff resources were limited. The time required to design, develop and deploy the conversational, AI-enabled virtual agents ranged from 2 to 14 days. The benefits of Watson Assistant are that it is able to use conversational AI through many channels, and it effectively handles voice in multiple languages. Table 2. Various applications of Watson Assistant in support of COVID-19 response during which users interacted via voice, text or the internet to ask questions about getting tested or accessing services during the pandemic. *CHOA data represents interactions from March 26 through June 3, 2020. **City of Austin, Texas and University of Arkansas for Medicine Sciences data represents interactions during April 2020. Organization Children’s Hospital of Atlanta* City of Austin, Texas** University of Arkansas for Medicine Sciences** Description of technology deployed in response to COVID-19 Pediatric COVID-19 Assessment Tool [23] City Information Chatbot [24] COVID-19 Screening Survey Agent [25] # unique conversations 10,414 15,998 3,514 # questions asked during conversation -- 24,378 7,480 # conversations where advice given or information resources provided 7,729 23,159 6,956 % conversations with a successful outcome 74% 95% 93% Design and implementation time 2 days 7 days in English; 14 days in Spanish 9 days Leveraging Informatics and Technology to Support Public Health Response: Framework and Illustrations using COVID-19 11 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 13(1):e1, 2021 OJPHI The first is The Children’s Hospital of Atlanta (CHOA), which implemented a COVID-19 Pediatric Assessment Tool to help parents answer questions pertaining to their child, such as what should be done if a child has a fever and/or cough?; and what should be done if a child may have been around someone with COVID-19 (exposed) but has no symptoms? The tool used decision tree analysis to evaluate a child’s symptoms (e.g. fever, marked lethargy, coughing, cyanosis). Importantly, the tool presented content at an appropriate reading level for a diverse lay audience. The virtual agent suggested next steps for a parent to take (e.g. advice to stay home, see a doctor, call 911) according to CHOA’s established protocols. The bilingual tool handled multiple channel mediums (e.g., English and Spanish text, English voice). The second is The City of Austin, Texas, the 11th largest city in the United States with more than 990,000 residents, which deployed a bilingual (English and Spanish) chatbot to respond to citizens. The 24x7 virtual chatbot facilitated questions and answers about the CDC’s COVID-19 recommendations, stay home orders, local screening availability, operating guidance and resources for businesses, job loss and unemployment services, homeless benefits including shelter and financial assistance, senior citizens, and building permits and inspections for developers. The third represents online and telephone screening for COVID-19 at the University of Arkansas for Medicine Sciences (UAMS) for adults aged 18 years and older residing in Arkansas who have no COVID-19 symptoms. IBM collaborated with two UAMS physicians to implement a pre-visit COVID-19 screening survey. Pregnancy screening questions and educational materials were also added. When the survey is completed by the user, the virtual assistant sends an email containing the answers to UAMS’ COVID-19 test center so that the patient information is waiting for them when they arrive to be tested. A mobile COVID-19 triage clinic helps to speed response. Depending on responses, an interactive video consult may be conducted with referral to appropriate response teams. Discussion We describe a framework designed to characterize how informatics and technology can support public health preparedness, recovery, and response during a pandemic. The framework maps the roles that informatics and technology play during a pandemic onto the major phases of a pandemic and the stakeholders involved. The framework further encapsulates the complexity of multiple sectors working together to respond and recover from a pandemic. Underlying this framework are three enabling forces. Leading businesses are investing in AI and cloud to predict future outcomes, create intelligent workflows that automate decisions and experiences, and empower people to do higher value work. Data and security are fueling digital transformation. AI and advanced analytics are unlocking the value of trusted data through insights. The convergence of these forces is catalyzing innovation in the creation of scalable cognitive solutions with intelligent interfaces to rapidly address disruptive change such as a pandemic. The COVID-19 pandemic highlights the importance of informatics to public health in an interconnected world and at the local level. The concept of Public Health 3.0 [8] requires a robust, interoperable infrastructure that supports data management and exchange between public health and other sectors. To fully achieve Public Health 3.0, patients, citizens, healthcare professionals, Leveraging Informatics and Technology to Support Public Health Response: Framework and Illustrations using COVID-19 12 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 13(1):e1, 2021 OJPHI public health officials, academics, and businesses all need to play a role in the development and use of health information technology and applications for pandemic preparedness, response and recovery. Investment in solutions for disease modeling for public health agencies are needed in the same way as solutions for working from home, selling goods and services from the cloud, and returning to the workplace. The examples given in this paper to illustrate framework components are works-in-progress, and they are just a few of the many technologies used for public health preparedness, response, and recovery against COVID-19. Evaluation studies of these and other approaches and technologies are required to fully understand their contributions to preparedness, recovery, and response for specific populations. Conclusion The framework illustrates how broad stakeholder groups across multiple sectors leverage technology and informatics during a pandemic. While this framework shows how each informatics methodology or tool fits into the landscape of public health preparedness, response, and recovery, studies are required to examine each tool and its contribution to a given phase of public health response or outcomes on specific populations. Understanding how these technologies performed, what worked, what was not effective and what could have worked better is critical to success in the next pandemic. Acknowledgments We would like to gratefully acknowledge the contributions and support of Sheri Bachstein, Carol Bales, and Melissa Medori from The Weather Channel and Edward Cadow, Judy Kelly, Preeth Muthusamy, Thomas Nisbet, Steve Payment, Paul Roma, Elizabeth Transier, Michael Volpe, and Leigh Williamson from Watson Health. No Financial Disclosures Competing Interests William Kassler, Hema Karunakaram, Kyu Rhee, and Jane Snowdon are employees of IBM Corporation. Brian Dixon has no competing interests. References 1. Obeid JS, Davis M, Turner M, et al. 2020. An AI approach to COVID-19 infection risk assessment in virtual visits: a case report. J Am Med Inform Assoc. 27(8), 1321-1325. doi:https://doi.org/10.1093/jamia/ocaa105. Epub 2020-07-04. 2. Paules CI, Eisinger RW, Marston HD, et al. 2017. What Recent History Has Taught Us About Responding to Emerging Infectious Disease Threats. Ann Intern Med. 167(11), 805- 11. doi:https://doi.org/10.7326/M17-2496. Epub 2017-11-14. PubMed https://doi.org/10.1093/jamia/ocaa105 https://doi.org/10.7326/M17-2496 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=29132162&dopt=Abstract Leveraging Informatics and Technology to Support Public Health Response: Framework and Illustrations using COVID-19 13 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 13(1):e1, 2021 OJPHI 3. Organization WH. Coronavirus disease (COVID-19) situation reports. 2020. https://www.who.int/emergencies/diseases/novel-coronavirus-2019/situation-reports. 4. Organization WH. Coronavirus disease (COVID-19) Situation Report — 160. 2020. https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200628-covid-19- sitrep-160.pdf?sfvrsn=2fe1c658_2 (accessed June 30, 2020). 5. Hurt B, Kligerman S, Hsiao A. 2020. Deep Learning Localization of Pneumonia: 2019 Coronavirus (COVID-19) Outbreak. J Thorac Imaging. 35(3), W87-W89. doi:https://doi.org/10.1097/RTI.0000000000000512. Epub 05-2020. PubMed 6. Mei X, Lee H-C, Diao K-y, et al. 2020. Artificial intelligence–enabled rapid diagnosis of patients with COVID-19. Nat Med. 35(3), W87-89. doi:10.1038/s41591-020-0931-3. Epub 2020-05-19. PubMed 7. Vaishya R, Javaid M, Khan IH, et al. 2020. Artificial Intelligence (AI) applications for COVID-19 pandemic. Epub 2020-04-14. Diabetes Metab Syndr. 14(4), 337-39. doi:https://doi.org/10.1016/j.dsx.2020.04.012. PubMed 8. DeSalvo K, Wang YC. Health Informatics in the Public Health 3.0 Era: Intelligence for the Chief Health Strategists. J. Public Health Manag. Pract. 2016;22 Suppl 6, Public Health Informatics (Suppl 6):S1-S2 doi: 10.1097/PHH.0000000000000484. Epub 2016-09-30. 9. DeSalvo KB, Wang YC, Harris A, et al. 2017. Public Health 3.0: A Call to Action for Public Health to Meet the Challenges of the 21st Century. Prev Chronic Dis. 14( E78). doi:https://doi.org/10.5888/pcd14.170017. Epub 09-2017. PubMed 10. Holloway R, Rasmussen SA, Zaza S, et al. 2014. Updated Preparedness and Response Framework for Influenza Pandemics. MMWR. 63(6). https://www.cdc.gov/flu/pandemic- resources/pdf/mmwr-rr6306.pdf. PubMed 11. Centers for Medicare & Medicaid Services (CMS) . Pandemic Influenza Operations and Response Plan, September 2009. https://www.cms.gov/About-CMS/Agency- Information/H1N1/Downloads/pandemicplan.pdf. 12. Getz KA, Wenger J, Campo RA, et al. 2008. Assessing the Impact of Protocol Design Changes on Clinical Trial Performance. Am J Ther. 15(5), 450-57. doi:https://doi.org/10.1097/MJT.0b013e31816b9027. PubMed 13. Getz KA, Stergiopoulos S, Short M, et al. 2016. The Impact of Protocol Amendments on Clinical Trial Performance and Cost Epub 2016-02-22. Ther Innov Regul Sci. 50(4), 436-41. doi:https://doi.org/10.1177/2168479016632271. PubMed 14. Lamberti MJ, Wilkinson M, Donzanti BA, et al. 2019. A Study on the Application and Use of Artificial Intelligence to Support Drug Development. Epub 2018-04-26. Clin Ther. 41(8), 1414-26. doi:https://doi.org/10.1016/j.clinthera.2019.05.018. PubMed https://doi.org/10.1097/RTI.0000000000000512 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=32205822&dopt=Abstract https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=32427924&dopt=Abstract https://doi.org/10.1016/j.dsx.2020.04.012 https://pubmed.ncbi.nlm.nih.gov/32305024/ https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=28880837&dopt=Abstract https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=25254666&dopt=Abstract https://doi.org/10.1097/MJT.0b013e31816b9027 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=18806521&dopt=Abstract https://doi.org/10.1177/2168479016632271 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=30227022&dopt=Abstract https://doi.org/10.1016/j.clinthera.2019.05.018 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=31248680&dopt=Abstract Leveraging Informatics and Technology to Support Public Health Response: Framework and Illustrations using COVID-19 14 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 13(1):e1, 2021 OJPHI 15. Wilkinson M, Young R, Harper B, et al. 2019. Baseline Assessment of the Evolving 2017 eClinical Landscape. Ther Innov Regul Sci. 53(1), 71-80. doi:https://doi.org/10.1177/2168479018769292 Epub 2018-04-26. PubMed 16. Reeves JJ, Hollandsworth HM, Torriani FJ, et al. 2020. Rapid response to COVID-19: health informatics support for outbreak management in an academic health system. J Am Med Inform Assoc. 27(6), 853-59. doi:https://doi.org/10.1093/jamia/ocaa037. Epub 2020- 06-01. PubMed 17. Douglas JV, Bianco S, Edlund S, et al. 2019. STEM: An Open Source Tool for Disease Modeling. Health Secur. 17(4), 291-306. doi:https://doi.org/10.1089/hs.2019.0018. Epub 07-2019. PubMed 18. Dixon BE, Grannis SJ, McAndrews C, et al. 2021. Leveraging data visualization and a statewide health information exchange to support COVID-19 surveillance and response: Application of public health informatics. J Am Med Inform Assoc. In Press. doi:https://doi.org/10.1093/jamia/ocab004. Epub 01-2021. PubMed 19. Flu season is here! Stay informed by using Flu Insights with Watson in The Weather Channel App. 2020. https://newsroom.ibm.com/Flu-season-is-here-Stay-informed-by-using- Flu-Insights-with-Watson-in-The-Weather-Channel-App. 20. Chukura R, Sulpovar M. How AI and Weather Data Can Help You Plan for Allergy Season, 2020. https://www.ibm.com/blogs/think/2020/04/the-question-this-spring-is-it-covid-or- allergies/. 21. Buscemi J. The Self-Assembled IBM Team Behind Two Widely Used COVID-19 Data Tracking Tools. 2020. https://newsroom.ibm.com/index.php?s=34178&item=31953. 22. IBM Offers Free Tools Based on Trusted Data to Track COVID-19 Cases on Your Phone and Online. 2020. https://newsroom.ibm.com/2020-03-25-IBM-Offers-Free-Tools-Based- on-Trusted-Data-to-Track-COVID-19-Cases-on-Your-Phone-and-Online. 23. CHOA COVID-19 Pediatric Assessment Tool. https://covid-choa.mybluemix.net/dashboard?view=chat. 24. City of Austin COVID-19 Free Assessment & Testing. https://covid19.austintexas.gov/s/?language=en_US. 25. Novel Coronavirus UAMS. (COVID-19) Screening. https://uams.virtriage.com/#/uamscovid19. https://doi.org/10.1177/2168479018769292 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=29714600&dopt=Abstract https://doi.org/10.1093/jamia/ocaa037 https://doi.org/10.1089/hs.2019.0018 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=31433284&dopt=Abstract https://doi.org/10.1093/jamia/ocab004 https://pubmed.ncbi.nlm.nih.gov/33480419/ Leveraging Informatics and Technology to Support Public Health Response: Framework and Illustrations using COVID-19 Abstract Introduction Methods Prepare Respond Recover A Framework for Pandemic Preparedness, Response, and Recovery Benchside Bedside Disease Investigation, Modeling and Monitoring Community Coordination and Programs Communications & Automated Assistance Return to Work Business Continuity and Resiliency Supply Chain Disruption New Ways of Working Mapping Results The Weather Channel COVID-19 Information Hub Watson Assistant for Citizens Discussion Conclusion Acknowledgments Competing Interests References