Development of a Process and Infrastructure to Outreach Stakeholders and Capture Healthcare System Stress in Emergency Response Situations Development of a Process and Infrastructure to Outreach Stakeholders for Capturing Healthcare System Stress in Emergency Response Situations Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(2):e2, 2019 OJPHI Development of a Process and Infrastructure to Outreach Stakeholders for Capturing Healthcare System Stress in Emergency Response Situations Taylor Read,1 Elizabeth White,2 J Perren Cobb,3 Perry Mar,4,5 Mahesh Shanmugam,6 Roberto A. Rocha,4,5 Sarah Collins Rossetti7* 1 Northwestern University, Chicago, IL, 2 Yale School of Public Health, New Haven, CT, 3 University of Southern California, Los Angeles, CA, 4 Brigham and Women’s Hospital, Boston, MA, 5 Harvard Medical School, Boston, MA, 6 Partners Healthcare System, Boston, MA, 7 Columbia University, New York, NY Abstract Real time data provided by frontline clinicians could be used to direct immediate resources during a public health emergency and inform increased preparedness for future events. The United States Critical Illness and Injury Trials Group Program for Emergency Preparedness (USCIIT-PREP), a group of expert critical care and emergency medicine physicians at various academic medical centers across the US, aims to enhance the national capability of rapid electronic data collection, along with analysis and dissemination of findings. To achieve these aims, USCIIT-PREP created a process for real-time data capture that relies on a curated and engaged network of clinical providers from various geographical regions to respond to short online “Pulse” queries about healthcare system stress. During a period of three years, five queries were created and distributed. The first two queries were used to develop and validate the data collection infrastructure. Results are reported for the last three queries between June 2015 and March 2016. Response rates consistently ranged from 39% to 42%. Our team demonstrated that our system and processes were ready for creation and rapid dissemination of episodic queries for rapid data collection, transmittal, and analysis through a curated national network of clinician responders during a public health emergency. USCIIT-PREP aims to further increase the response rate through additional engagement efforts within the network, to continue to grow the clinician responder database, and to optimize additional query content. Keywords: Public Health Informatics, Emergency preparedness, electronic data capture, real-time data capture Abbreviations: United States Critical Illness and Injury Trials Group Program for Emergency Preparedness (USCIIT-PREP); Office for the Assistant Secretary for Preparedness and Response (ASPR); World Health Organization (WHO); Research Electronic Data Capture (REDCap) Correspondence: * Roberto A. Rocha, r.rocha@computer.org. * Sarah Collins Rossetti, sac2125@cumc.columbia.edu DOI: 10.5210/ojphi.v11i2.10048 Copyright ©2019 the author(s) This is an Open Access article. Authors own copyright of their articles appearing in the Online Journal of Public Health Informatics. Readers may copy articles without permission of the copyright owner(s), as long as the author and OJPHI are acknowledged in the copy and the copy is used for educational, not-for-profit purposes. http://ojphi.org/ r.rocha@computer.org mailto:sac2125@cumc.columbia.edu Development of a Process and Infrastructure to Outreach Stakeholders for Capturing Healthcare System Stress in Emergency Response Situations Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(2):e2, 2019 OJPHI Introduction The United States lacks a technical infrastructure that allows “frontline” clinical providers to collect and report real-time clinical and population data during an emergency. For example, during an influenza outbreak, understanding a data-driven level of stress incurred at healthcare institutions might facilitate provision of resources and guide subsequent planning. Real time data could be used to direct immediate resources and inform increased preparedness for future events [1]. Stressed institutions are defined as those having to implement alternative policies and procedures to manage day-to-day operations. A public health emergency is a critical period and the process of collecting data during this time should not place undue burden on clinical providers, be delayed in execution, or lack methodological rigor that prevents comparisons over time [2]. As an expedient, scalable, sustainable, and repeatable approach to collect real-time data during a public health emergency, our team has created a network of clinician responders from across the country committed to responding to rapid data collection queries. We aimed to use rapid data collection queries to take the ‘pulse’ of the healthcare system within 24 hours of an identified potential stressor, particularly a public health emergency. The United States Critical Illness and Injury Trials Group Program for Emergency Preparedness (USCIIT-PREP) is a group of expert critical care and emergency medicine physicians at various academic medical centers across the US. USCIIT-PREP aims to significantly enhance the national capability of rapid electronic data collection, along with analysis and dissemination of findings [1]. The process created by the USCIIT-PREP “Pulse Project” for real-time data capture relies on a curated and engaged network of clinical providers from various geographical regions to respond to online queries about healthcare system stress [3]. Pulse queries are intended to be short and capture only critical data, providing a snapshot into challenges that each site might be experiencing. By collecting data snapshots covering multiple geographical regions, analysis can be rapidly performed and disseminated to different stakeholders. The Pulse Project was funded through a contract from the Office for the Assistant Secretary for Preparedness and Response (ASPR), whom directs the content and dates of each prospective query. ASPR was created after Hurricane Katrina to focus on prevention, preparation, and response to public health emergencies and disasters [2]. ASPR provides operational response capabilities, medical countermeasure research, as well as grants for health care organizations to strengthen readiness procedures [4]. We describe here the first phase of the Pulse Project, from March 2014 to March 2016, which was primarily focused on process and infrastructure development and validation. Background A public health emergency can come from various sources, including but not limited to environmental disasters, pandemics/epidemics, and resource shortages. When these issues arise, it is vital to have a response system in place ready to assess and react with proper protocol and the necessary measures to deescalate the situation. The needs of the healthcare system during a public health emergency should be supported by data-driven solutions, particularly outcomes confirmed by data analysis from prior events [5]. As emergency conditions progress, needs can http://ojphi.org/ Development of a Process and Infrastructure to Outreach Stakeholders for Capturing Healthcare System Stress in Emergency Response Situations Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(2):e2, 2019 OJPHI be anticipated, and countermeasures initiated by evaluating trends from previous emergencies. A delayed response to an outbreak may be attributed to multiple factors, including disease or condition-specific variables, political conditions, economic support, or environmental conditions [6]. Despite the challenges associated with real-time data collection, such as rapid configuration and dissemination of electronic data capture tools, it is important to strive for concurrent data collection, as opposed to retrospective, in order to identify opportunities to quickly provide assistance [5]. Per the World Health Organization (WHO), there is no standardized protocol for data collection and sharing during public health emergencies [7]. These shortcomings have been increasingly apparent during recent public health emergencies, like the 2009 H1N1 pandemic. Collection of vital clinical data to synthesize treatments and identify high-risk groups remained challenging [2]. WHO has started to create methods for public data sharing, but these practices have not been tested or validated in the public health realm. Other groups have created data collection tools in response to specific outbreaks, such as Ebola, but these were created after the outbreak and therefore lack critical data [8]. During the recent Zika outbreak, the WHO released a data-sharing platform upon the declaration of the Public Health Emergency of International Concern in 2016, allowing researchers from around the world to share and access data about the outbreak [9]. In a separate effort, our team created a tool to assess nationwide variability in treatment of influenza for critically ill patients [10]. Methods During this first phase of the Pulse Project, the main goals were to implement and validate a reliable and consistent process for rapid data collection during a real or perceived public health emergency, and to transmit and analyze the resulting data to inform and enable an appropriate response [11]. We focused on three critical infrastructure components needed to adequately prepare for real-time data collection during an emergency: a) an easily accessible electronic data collection tool, b) a network of clinician responders tailored to the specificity of the stressor and area of concern, and c) clinically relevant and validated query instruments. Considering the overarching rapid data collection process and these three infrastructure components, we identified four requirements and three associated challenges prior to launching a new query. In terms of requirements, each query had to be (i) concise (i.e. take only a few minutes to respond), (ii) clear (i.e. easily understandable), (iii) remain open for a short period of time (i.e. 48-hour period), and (iv) distributed to an adequate number of clinician responders in multiple regions of the country. Challenges in meeting these requirements included (v) enabling the rapid design of a query instrument relevant to the specific public health emergency, (vi) quickly disseminating the query to the representative target audience (i.e. experienced clinical providers available and willing to respond), and (vii) quickly identifying alternate responders for those that were unable to respond. The Pulse Project aimed to create a configurable data collection infrastructure that could be thoroughly validated before a potential public health emergency. The configurability ensured the rapid creation and dissemination of different data collection instruments to clinical providers http://ojphi.org/ Development of a Process and Infrastructure to Outreach Stakeholders for Capturing Healthcare System Stress in Emergency Response Situations Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(2):e2, 2019 OJPHI using the Internet. Necessary technical components included a software tool to create and manage queries, a database of clinician responders, as well as web-based analytics tools and geographical mapping software for concurrent data analysis and visualization [12]. The Pulse Project adopted the ten geographical regions outlined by the U.S. Department of Health & Human Services [13]. Each region had one or two appointed leads serving as the point person for the clinician responders in that area. The responsibilities of regional leads included outreach and communication to clinician responders. In order to increase the likelihood of a response, prior to each query, our team sent brief requests to clinician responders. These pre- query requests included a confirmation of availability to respond to queries, but also the validation of prior collected demographic details, hospital bed count, and academic and clinical (department) affiliation. Query implementation Pulse queries were implemented using the Research Electronic Data Capture (REDCap), a secure web-based application developed for electronic data capture [14]. However, REDCap had to be extended to meet our data collection and analysis requirements. For example, we configured a mapping application within REDCap to generate real-time maps with response rates and geographical distributions, taking into account the location of the various clinician responders. Another extension was the ability to prefill query fields with demographic information previously provided by registered clinician responders. Prefilling demographic details reduced the effort required from clinician responders, an important feature to reduce the burden of real- time data capture during public health emergencies. Our team created and validated a reusable set of standard, generic questions that could apply to several types of public health emergencies. This set of generic questions was validated and optimized with each subsequent query, addressing different aspects of staffing, resources, capacity, and utilization of high-acuity protocols. These same set of generic questions was used for planned quarterly queries, which assessed stress levels at periodic points throughout the year. However, when a specific type of stressor or event is identified, the set of generic query questions was supplemented with additional questions tailored to the specific stressor. These additional stressor-specific questions were rapidly identified and validated, in collaboration with subject matter experts. For example, a query during an influenza outbreak included questions about viral testing procedures, while another query during a saline shortage included questions about alternative practices and products being utilized. While quarterly queries were planned to collect stress data periodically throughout the year, our team was also prepared to send episodic queries when potential stressor events were identified. Only a few episodic queries were sent throughout the project’s duration. After the design of a query instrument was finalized, the data capture system was promptly configured, and the online instrument was distributed to the network of clinician responders. In an effort to minimize misinterpretations across different regions and institutions, query instruments only used clear and concise language that conveyed consistent and unambiguous http://ojphi.org/ Development of a Process and Infrastructure to Outreach Stakeholders for Capturing Healthcare System Stress in Emergency Response Situations Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(2):e2, 2019 OJPHI questions as determined during validation sessions with subject matter experts. Each query started at 8:00 AM Eastern Standard Time, and closed 48 hours later. A period of 48 hours provided adequate time for clinician responders to answer the brief query after determining their local stress at that particular point in time. Once a query was initiated, it was crucial to rapidly analyze the data being collected. An initial analysis phase occurred while queries were still running, using descriptive statistics and focusing on response rates by region. Response rates were calculated periodically during the 48-hour period, and sent to regional leads to inform the performance of their region when compared to others. Regional leads with low response rates were asked to reach out to clinician responders in their region and encourage participation. Query data analysis Once a query was closed, all collected data were sent to ASPR stakeholders who performed the data analysis at a state, regional, and national level. All data collected were considered confidential and promptly transferred to ASPR. At its discretion, ASPR analyzed and communicated the actual findings to stakeholders. Our group utilized the collected data during a query to evaluate the query instrument and the overall data collection process, including specific levels of participation. These details were used to optimize subsequent queries. The post-query analysis evaluated the distribution of responders’ hospitals by size and category, using bed count and the Medicare cost report, respectively. Each hospital was identified within the American Hospital Directory to obtain bed count and other details [15]. We also determined the geographical distribution of respondents. “Point maps” were created to identify the cities of participating hospitals with different map overlays. One overlay used region boundaries, enabling regional leads to assess areas within their region that lacked representation. Another overlay was color-coded with population density from U.S. census data (Figure 1). The map overlay with population density helped us prioritize outreach efforts to specific geographical locations, particularly to ensure proper representation of areas with higher population density. These maps were generated in real-time within REDCap, during and after each query. http://ojphi.org/ Development of a Process and Infrastructure to Outreach Stakeholders for Capturing Healthcare System Stress in Emergency Response Situations Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(2):e2, 2019 OJPHI Figure 1 – Example of a map overlay with population density. Each dot represents a city with participating clinician responders. Results During a period of three years, five queries were created and distributed. We report here the results of the last three queries, which occurred between June 2015 and March 2016 (Table 1). The first two queries, which occurred between March 2014 and April 2015, were preliminary efforts used to develop and validate the data collection infrastructure, as well as build the network of clinical responders [2]. Pulse query ‘G1’ used the set of generic questions and included two pre-query requests. The first request was used to obtain confirmation that clinical responders were available to participate. The second request was actually a “mock” query sent one week prior to the actual query. Pulse query ‘G2’ also included two pre-query requests. The first pre-query request was sent three weeks prior to the query and was used to obtain additional clinician responder details, such as phone number, system name, hospital name, clinical department, and role within the department. This pre-query request also included, for the first time, a section for clinician responders to designate a back-up contact within their department. The intent was to identify another clinical provider that could respond on their behalf if they were ever unavailable to respond. The second ‘G2’ pre-query request was sent one week before the query, with a single question confirming their participation. This participation confirmation also provided an opportunity to validate the alternate contact identified during the previous request. Pulse query ‘I1’ was able to reuse the information collected during previous queries and did not include any pre-query requests. Instead, while G1 and G2 queries were planned, query ‘I1’ simulated a true emergency and was promptly setup and distributed. http://ojphi.org/ Development of a Process and Infrastructure to Outreach Stakeholders for Capturing Healthcare System Stress in Emergency Response Situations Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(2):e2, 2019 OJPHI Table 1 – Pulse query details between June 2015 and March 2016 Pulse query number Date issued Type of query (focus) Query recipients Query respondents Response Rate G1 Jun 2015 General healthcare system stress 344 141 40.99% G2 Dec 2015 General healthcare system stress 385 161 41.82% I1 Mar 2016 Influenza- related healthcare system stress 354 138 38.98% Pre-query requests were analyzed to confirm response rates and correlate to subsequent query response rates. For instance, the pre-query request sent prior to Pulse query ‘G2’, 69 out of 385 (17.9%) providers responded with their personal contact information. Of those, 32 (46.4%) provided a back-up contact within their department to respond if they were unavailable. The participation confirmation request sent a week before query ‘G2’ generated 103 responses, including 96 positive answers from clinical responders confirming participation. Out of the seven negative answers, five clinical responders also included a back-up contact. Of the 96 clinician responders who confirmed participation, 85 (88.5%) actually responded to Pulse query ‘G2’. Of the 282 responders who disregarded the participation confirmation request, 73 (25.9%) responded to Pulse query ‘G2’. Three (60.0%) clinical providers designated as back-up responded to query ‘G2’, out of the five alternate contacts used. Regional leads were asked to oversee the participation of local clinical responders. Each lead received a regional analysis during the query to help identify data collection gaps. Analysis of regional response rates during Pulse query ‘G2’ shows four regions having a greater than 50% response rate, three regions between 30-50%, and three regions below 30% (Table 2). We confirmed that regional lead outreach to clinical responders increased response rates, as demonstrated by response rate differences between regions with and without active outreach. http://ojphi.org/ Development of a Process and Infrastructure to Outreach Stakeholders for Capturing Healthcare System Stress in Emergency Response Situations Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(2):e2, 2019 OJPHI Table 2 – Pulse query ‘G2’ response rates over time by region Region After 24 hours After 33 hours Final (48 hours) 1 39% (14/36) 56% (20/36) 58% (21/36) 2 29% (10/35) 31% (11/35) 31% (11/35) 3 35% (11/31) 39% (12/31) 48% (15/31) 4 30% (14/47) 51% (24/47) 55% (26/47) 5 25% (14/55) 47% (26/55) 56% (31/55) 6 19% (4/21) 24% (5/21) 24% (5/21) 7 11% (3/27) 15% (4/27) 19% (5/27) 8 13% (3/23) 26% (6/23) 30% (7/23) 9 13% (10/77) 22% (17/77) 29% (22/77) 10 24% (7/29) 41% (12/29) 59% (17/29) The size of the hospitals that responded to Pulse queries ‘G1’ and ‘G2’ are summarized in Table 3, taking into account total number of beds and number of critical care beds. Large hospitals, based on the American Hospital Directory, are defined as having 400 beds or more, medium with 100 to 399 beds, and small hospitals with 100 beds or less. The distribution by type included 116 academic medical centers, 31 teaching hospitals, 3 critical access hospitals, 9 regional hospitals, 5 county hospitals and 2 corporate hospitals. Table 3 – Hospitals that responded to Pulse queries ‘G1’ and ‘G2’ grouped by size Bed count Small Hospitals Medium Hospitals Large Hospitals All beds: range < 100 100-399 > 400 Critical care: range 0-36 7-153 28-409 Critical care: median 0 35 136 Critical care: mean 5 45 145 Number of hospitals 10 41 106 Discussion Our team demonstrated successful implementation and validation of a reliable and consistent process for rapid data collection, transmittal, and analysis during a public health emergency. Specifically, we deployed an easily accessible electronic data collection tool to an engaged and http://ojphi.org/ Development of a Process and Infrastructure to Outreach Stakeholders for Capturing Healthcare System Stress in Emergency Response Situations Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(2):e2, 2019 OJPHI curated national network of clinician responders to capture clinically relevant data using validated query instruments. The Pulse project lasted three years and executed five different pulses during that time. The principle goals were to be prepared for rapid data accumulation during public health emergencies and to analyze resulting data to inform and enable responses with necessary interventions. To determine the emergency preparedness of the healthcare system, the pulse system tested responses to identified stressors by answering data collection queries. Through analysis of response rates, we were able to determine a consistent response rate for the Pulse queries, ranging from 39% to 42%. The slight increase in responses between Pulse G1 and Pulse G2 could be attributed to extensive outreach during the fall of 2015. The slight decline in participation from Pulse G2 to Pulse I1 (-3%) could be attributed to the lack of pre-query communication for Pulse I1. Our distribution of responses primarily remained consistent among the three Pulses with slight variation between states lacking participation. Having the regional leads send reminders during the Pulse query appears to positively influence the response rate. This data was shared with regional leads to provide evidence of their work and encourage future outreach to their region. Reviewing pre-query request processes allowed us to refine the process for future queries. After initial testing we removed test queries to avoid confusion but continued to send pre-query requests for general healthcare system stress queries. However, we did not send a pre-query request for Pulse I1 as it was immediately disseminated upon completion, aligned with an unplanned stress even that did not allow time for a pre-query. We aimed to increase response rates for pre-query requests in order to add additional information for each clinician responder into the database, which in return increases the possible number of pre-populated fields. By obtaining additional information before a query, we are able to determine unit of analysis for each clinician responder with department data as well as send queries to alternative clinicians if back-up clinicians are provided. Maintaining an up to date database ensures our response rates were accurate by removing clinicians who were no longer interested and replacing clinicians as they moved to different institutions. Using response rate data, we were able to stratify response rates by regions to identify areas of low response. Using these data we were able to provide assistance to regional leads directly as well as connecting regional leads to one another for additional bandwidth. Targeted outreach initiatives increased geographical representation across each region. Our hourly analyses indicated that response rates most likely increased due to the regional lead reminders sent during the queries as indicated by a sharp increase in responses after regional lead contact was made. As a result, we have included this as a formal process during all Pulse queries going forward. Looking at hospital size distribution, the overall goal was to use this data to create a less skewed distribution of hospitals in the Pulse database since the database initially favored large, academic medical centers. This involved seeking out smaller hospitals as well as county and critical access hospitals across the country. This will be a continuous challenge as our current method of recruitment involves networking with current clinician responders and regional leads. Review of http://ojphi.org/ Development of a Process and Infrastructure to Outreach Stakeholders for Capturing Healthcare System Stress in Emergency Response Situations Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(2):e2, 2019 OJPHI geographical hospital distribution provided tiered regions for outreach. Our first priority is to perform outreach to states with zero responses, then states with one response, followed by states represented by a single metropolitan area. We utilized the network of network approach by reaching out to current responders asking for referrals to clinical providers in the regions of interest. Limitations This was a nationwide study lending strength to the sample size and network of clinician responders utilized. However, we acknowledge that a larger and more diverse sample of pulse queries, particularly those in response to an unanticipated event, would strengthen our understanding of the generalizability, scalability and replicability of our results to date. Specifically, we note that influenza is an annual occurrence. The results would be strengthened further with data from a pulse query to an unanticipated event that does not allow for pre-query requests and requires substantial supplementation with additional questions tailored to the specific stressor. The time to validate a query instrument requiring significant additions may be substantial and does pose a challenge for executing real-time queries. We believe our approach in using a standard, generic form that can be extended as needed minimizes, but does not completely remove, that effort. Further, the curated network is comprised of critical care and emergency medicine physicians and it is possible that an unanticipated event may require development and curation of a network with different clinical expertise. Finally, there was lower representation from county and critical access hospitals across the country and variation in participation across states. Further work is required to engage these populations for a more representative sample of system stress. Conclusion We showed that the process and tools used throughout this project were able to rapidly execute expedient Pulse queries through a pre-configured infrastructure and established network of clinician responders. We achieved reliable response rates ranging from 39% to 42% despite the variety in pre-query communication and emergent nature of a query. We determined outreach processes to be successful at maintaining the Pulse database of clinician responders due to pre-query requests providing additional useful database fields. Distributional analysis by hospital size, type and location was useful for optimization of future outreach to create a more representative database of clinician responders. With several Pulse queries completed successfully, the process is ready for future queries. The I1 query that was configured in a time sensitive setting, demonstrated that our system and processes were ready for creation and rapid dissemination of an episodic query if a public health emergency were to arise and ongoing work has confirmed this continued success [16]. USCIIT-PREP aims to further increase the response rate by engaging the regional leads to send personalized reminders to non-responders during the query period as well as establishing pre- query communication with all clinician responders. As of August 2016, all Pulse related initiatives have transitioned to University of Southern California per the ASPR contract [16]. http://ojphi.org/ Development of a Process and Infrastructure to Outreach Stakeholders for Capturing Healthcare System Stress in Emergency Response Situations Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(2):e2, 2019 OJPHI USCIIT-PREP will continue to grow the clinician responder database using distribution data such as hospital size, type and geographical location. Overtime, we plan to have optimized several sets of query questions that can be reused depending on the type of query. Funding acknowledgements: Funding provided through a contract from the Office for the Assistant Secretary for Preparedness and Response (ASPR). References 1. Blum JM et al. United States Critical Illness and Injury Trials Group. 2013. Chest. 143(3), 808-13. PubMed https://doi.org/10.1378/chest.12-2287 2. Lurie N, Manolio T, Patterson A, Collins F, Frieden T. 2013. Research as a Part of Public Health Emergency Response. N Engl J Med. 368(13), 1251-55. PubMed https://doi.org/10.1056/NEJMsb1209510 3. Read T, White E, Karipineni N, Ignatov M, Shanmugam M, et al. 2016. Utilization of Cost Effective Tools for Queries on Healthcare System Stress. AMIA Annu Symp Proc. 1564. 4. ASPR Organization. https://www.phe.gov/about/aspr/Pages/default.aspx. Accessed September 16, 2018. 5. Morton M, Levy JL. 2011. Challenges in disaster data collection during recent disasters. Prehosp Disaster Med. 26(3), 196-201. PubMed https://doi.org/10.1017/S1049023X11006339 6. Lee EK, Liu Y, Ferdinand PH. 2017. A Compartmental Model for Zika Virus with Dynamic Human and Vector Populations. AMIA Annu Symp Proc. 2016, 743-52. PubMed 7. Whitty CJ, Mundel T, Farrar J, Heymann D, Davies S, et al. 2015. Providing incentives to share data early in health emergencies: the role of journal editors. Lancet. 386(10006), 1797- 98. PubMed https://doi.org/10.1016/S0140-6736(15)00758-8 8. Sacks JA et al. 2015. Introduction of Mobile Health Tools to Support Ebola Surveillance and Contact Tracing in Guinea. Glob Health Sci Pract. 3(4), 646-59. PubMed https://doi.org/10.9745/GHSP-D-15-00207 9. World Health Organization - Zika Open. http://www.who.int/bulletin/online_first/zika_open/en/. Accessed July 27, 2018. 10. 2017. Discovery Program for Emergency Preparedness. Variability in Treatment of Influenza in the Critically Ill: A 2017 Cohort Study. Crit Care Med. 46(1), 314. http://ojphi.org/ https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23460158&dopt=Abstract https://doi.org/10.1378/chest.12-2287 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23534565&dopt=Abstract https://doi.org/10.1056/NEJMsb1209510 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22107771&dopt=Abstract https://doi.org/10.1017/S1049023X11006339 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=28269870&dopt=Abstract https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=26843294&dopt=Abstract https://doi.org/10.1016/S0140-6736(15)00758-8 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=26681710&dopt=Abstract https://doi.org/10.9745/GHSP-D-15-00207 Development of a Process and Infrastructure to Outreach Stakeholders for Capturing Healthcare System Stress in Emergency Response Situations Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(2):e2, 2019 OJPHI 11. 2018. Discovery Program for Emergency Preparedness. Electronic Data Capture During Public Health Emergencies. Crit Care Med. 46(1), 39. 12. Murphy DJ et al. 2015. Development of a Core Clinical Dataset to Characterize Serious Illness, Injuries, and Resource Requirements for Acute Medical Responses to Public Health Emergencies. Crit Care Med. 43(11), 2403-08. PubMed https://doi.org/10.1097/CCM.0000000000001274 13. Regional Offices. https://www.hhs.gov/about/agencies/iea/regional-offices/index.html. Accessed August 25, 2018. 14. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, et al. 2009. Research electronic data capture (REDCap) - a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 42(2), 377-81. PubMed https://doi.org/10.1016/j.jbi.2008.08.010 15. American Hospital Directory - information about hospitals from public and private data sources including MedPAR, OPPS, hospital cost reports, and other CMS files. https://www.ahd.com/. Accessed August 25, 2018. 16. 2018. Discovery Program for Emergency Preparedness, Assistant Secretary for Preparedness and Response. Longitudinal Assessment of U.S. Health System Stress During Influenza Seasons: Pulse Program. Crit Care Med. 46(1), 36. http://ojphi.org/ https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=26308434&dopt=Abstract https://doi.org/10.1097/CCM.0000000000001274 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=18929686&dopt=Abstract https://doi.org/10.1016/j.jbi.2008.08.010