Successful Public Health Information System Database Integration Projects: A Qualitative Study Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(2):e207, 2018 1 OJPHI Successful Public Health Information System Database Integration Projects: A Qualitative Study Matthew Roberts, MPH, DrPH School of Public Health, University of Illinois, Chicago, IL, USA Abstract Objective: To explore the most important public health information system database integration project success factors to include: technological, organizational, project-specific, or external. Methods: This study involved a cross-case design. Cases were identified through literal replication logic and screened through a survey and review of available literature. Study participants were interviewed through hour-long sessions steered by a semi-structured guide. Survey responses, interview transcripts and available documents were coded and analyzed deductively, and matrices were developed to illustrate relationships. Results: Leadership among the project’s participants is the most important integration project success factor. This leadership factor manifests in the following ways. Executive sponsors champion the initiative. Informaticians facilitate communication and system requirement collection. Program directors contribute substantive energy to the project and remove obstacles. Some other factors also contribute to project success. For example, strong Financial Management and Support promotes project initiation. Technological aspects impact the final product’s quality. Utilizing formal project management techniques, particularly the Agile software development methodology, contributes to successful project resolution by ensuring daily operational effectiveness. Discussion: The principal finding illustrates important contributions by project leaders, transcending those of the executive sponsor. Other participants, notably informaticians and program directors, substantially contribute to the project’s success. Additionally, the Agile software development methodology is emerging as a successful approach to project management for these and related projects. Conclusion: Investing in the leadership and project management skills of database integration project participants could improve the success of future projects. State health department staff considering these projects should carefully select project participants and train them accordingly. Keywords: Public Health Information Systems, Leadership, Systems Integration, Agile, Agile Methodology Abbreviations: Public Health Information Systems (PHIS), Child Health Information System (CHIS), Association of State and Territorial Health Officials (ASTHO), Informatics Directors Peer Network (IDPN), Information Technology (IT), National Electronic Disease Surveillance System (NEDSS) Correspondence: Matthew Roberts, University of Illinois at Chicago, 217-415-7931, matthew.wesley.roberts@gmail.com DOI: 10.5210/ojphi.v10i2.9221 mailto:matthew.wesley.roberts@gmail.com Successful Public Health Information System Database Integration Projects: A Qualitative Study Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(2):e207, 2018 2 OJPHI Copyright ©2018 the author(s) This is an Open Access article. Authors own copyright of their articles appearing in the Online Journal of Public Health Informatics. Readers may copy articles without permission of the copyright owner(s), as long as the author and OJPHI are acknowledged in the copy and the copy is used for educational, not-for-profit purposes. Introduction Public Health Information Systems (PHIS) are foundational components of public health infrastructure, providing how health departments collect and maintain data for public health practice [1]. These data support population health services such as controlling outbreaks or designing health promotion programs targeting teen smokers. State governments often establish PHIS through the state health department, and the systems primarily serve state and local health department data needs [2]. The use of information technology to develop databases is a critical aspect of PHIS. These databases store public health data, and advances in information technology have improved the ability to develop databases that suit specific program requirements [3]. Specialized state public health databases have propagated because of this technical development ease and categorical funding incentives. Computing advancements have also readily allowed for the integration of separate databases [4]. Database integration often entails the development of a common database for the organization that consolidates operational data from multiple sources [5]. When individual databases are integrated, they collectively create more complete records by piecing together different data elements from different sources. Access to complete records can improve coordination of public health activities and reduce costs [6]. The Joint Council of Governmental Public Health Agencies suggest that 1) health departments must integrate databases and 2) these integrated databases must meet information needs at the service delivery level [7]. Previous child health information system (CHIS) database integration activities illustrate the following: the development of a specific PHIS database integration business case, actions required to successfully execute the project, and prior integration project evaluation efforts [8]. A large measles outbreak in the late 1980s prompted public health and healthcare leaders to evaluate data collection and usage techniques, which led to the initiation of CHIS database integration efforts. A workgroup identified programs such as immunizations and vital registration as a suitable starting point for the integration projects [9]. Evaluation activities included documenting and studying the critical success factors for these integration projects. Findings from CHIS integration studies informed research in related areas [10]. Customized program-specific databases have proliferated but often they have not integrated with other databases throughout the health department. Many public health program managers have established databases without considering broader database integration. These databases meet the individual program’s data needs, but do not address enterprise information management needs across the organization [11]. Silo public health databases result in inefficiencies, such as poor disease control and outbreak response coordination; incomplete service delivery at the local level; and underperforming population health protection measures during public health emergencies [12]. While leaders integrated and evaluated some CHIS databases, few other successful PHIS Successful Public Health Information System Database Integration Projects: A Qualitative Study Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(2):e207, 2018 3 OJPHI database integration initiatives have been studied. It is not known which PHIS database integration project factors are most important and how they impact successful public health database integration projects. This study explored factors that most contribute to successful intra-state public health information system database integration projects. Technological factors, such as the quality of data within the originating data systems, impact the functionality of the integrated database. Organizational factors, such as meaningfully engaged executive leadership and strategic plans, impact the agency’s readiness and commitment to the initiative. Project-specific factors, such as effective governance and formal project management techniques, impact the day-to-day administration of the initiative. Other factors outside the agency’s direct control, such as data privacy laws or the actions of external stakeholders can also influence the project’s success. The Health Resources & Services Administration’s Sourcebook lists the nine non-technical integration project elements (factors) [13]. Table 1 includes this list in addition to the technical factors. All factors have been grouped into logical Domains for this study based upon prior research [14-21]. This manuscript will also describe how Agile software development facilitated the daily project management for many of the PHIS database integration initiatives illustrated in this study. The use of the Agile software development methodology in public health practice is poorly understood. Agile software development, or simply “Agile”, is emerging as a popular software development project management alternative to more traditional approaches such as the widely-used waterfall methodology. The Waterfall model entails a prescriptive stage-oriented software development process characterized by exhaustive initial requirement collection and design phases [22]. Agile is considered a “lightweight” method for developing software, with principles that focus on intensive collaboration and rapid software iteration versus extensive up-front system requirement documentation and highly-regimented planning [23]. Many technology companies utilize Agile to rapidly iterate software products and gain a competitive advantage. Organizations have utilized Agile to create software for healthcare applications [24], and others have modified aspects of the organizational culture by adopting Agile practices for managing other types of projects [25]. Researchers have studied their experiences in utilizing Agile to create and maintain biomedical software, and found the Agile approach to be a good fit for these projects [26]. Following the failed rollout of HealthCare.gov, some departments of the United States federal government immersed themselves in Agile methodology with some success [27]. Implementing the Agile methodology does not come without its risks for failure, but its success factors have been studied [28]. The role of Agile in the PHIS database integration projects identified in this study will be illustrated further. Table 1. Integration Project Success Factors, Grouped ORGANIZATIONAL DOMAIN Leadership The project has an executive sponsor, a high level official who advocates for the project, and a champion, someone who is willing to devote a significant effort to see the project succeed. Organizational and Technical Strategy Successful Public Health Information System Database Integration Projects: A Qualitative Study Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(2):e207, 2018 4 OJPHI The project has a strategy that takes into consideration local issues such as funding, the political environment, organizational structure, the strengths of the organization, and stakeholder beliefs and values. The selected technical integration approach accounts for internal data governance and data sharing needs, which must conform to state and federal laws and agreements made with stakeholders. PROJECT ORIENTED DOMAIN Project Governance The project is guided by a steering committee representing all key stakeholders. The steering committee develops the integration strategy, based on clearly defined business processes. Project Management The project has formalized management strategies and project management methodologies designed to assure consistent communications, accountability, and resource constraints. Technical Support and Coordination Technical information systems support and coordination is organized centrally to assure consistent support and a robust infrastructure capable of maintaining and complying with standards. A business analyst supports implementation. Financial Support and Management The project is adequately funded and has multiple funding sources. Evaluation The project has some form of qualitative and/or quantitative monitoring or evaluation that is performed regularly. EXTERNAL DOMAIN Stakeholder Involvement Frequent communication with stakeholders and involvement of stakeholders in the integration project throughout the life cycle of the project contributes to its success and credibility. Policy Support Rules, regulations, legislation, and policy advisory or policymaking bodies are supportive or at least neutral to the integration of health information systems. Executive sponsors educate policymakers about sensitive issues to garner their support. TECHNICAL DOMAIN Source Systems Databases contain quality program-specific data to be contributed to the database integration project. Development Technology Project managers select a particular technology to be utilized for the integration project including architecture, hardware, database software, data integration engines, user interface, etc. Successful Public Health Information System Database Integration Projects: A Qualitative Study Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(2):e207, 2018 5 OJPHI Methods A cross-case study design was used for this research. The state health department is the unit of observation, and the database integration projects are the unit of study. Case Selection The researcher identified cases through literal replication logic, seeking successful state health department intra-state public health information system database integration projects [29]. A three- phase screening procedure was utilized, entailing 1) deploying a survey; 2) reviewing successful PHIS database integration projects; and 3) reputational case selection. The survey targeted state health department informatics staff as members of the Association of State and Territorial Health Officials (ASTHO) Informatics Directors Peer Network (IDPN). The survey was administered to IDPN members, and the first phase of screening reduced the candidate list to cases that most closely fit the literal replication design. Seven participants responded to the survey, and three met the criteria for additional follow-up. Next, the researcher identified and screened cases based on criteria specified through a literature review of frequently documented, successfully completed PHIS database integration projects. Peer reviewed literature, books, and federal agencies have documented these projects, and this literature illustrates best practices and exemplars. In particular, CHIS database integration research and studies evaluating the Environmental Public Health Tracking activities provide substantive insight into successful PHIS database integration initiatives [30]. The researcher contacted representatives from state health departments presented in these research bodies for inclusion as participants in this study. Three cases were selected through this literature review process. Finally, one case was identified through reputational case selection referral by the Public Health Informatics Institute. Through these processes, the researcher retained seven cases that best fit the literal replication design of successful PHIS database integration projects. The study protocol was reviewed and approved by the University of Illinois – Chicago Institutional Review Board. Survey and interview questions The researcher adapted questions from a previous CHIS database integration study [31]. The survey questionnaire addressed case demographics and questions that pertain to the technical and project planning domains. The interview guide was piloted with four informaticians from state health departments, and the final version was organized into the Domain groupings from Table 1. The interview guide asked participants about the agency’s informatics projects and the impact of each Domain’s factors on the integration project’s success. Procedure The researcher conducted and recorded approximately one-hour semi-structured interviews with state health department informatics directors along with referral follow-up interviews of program directors, bureau chiefs, system administrators, and technical staff. The researcher interviewed twenty-five participants through nineteen interviews (some interviews included two participants) from April to September of 2016. The survey responses were then paired with associated interview transcripts. Finally, the researcher obtained from participants and websites copies of pertinent Successful Public Health Information System Database Integration Projects: A Qualitative Study Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(2):e207, 2018 6 OJPHI project documentation for review: strategic information technology plans and operational plans for the database integration projects; agency budgets; project meeting minutes; project charters; progress reports; policy documents; and protocols. Analyses The analysis included within-case analysis followed by cross-case analysis. The within-case analysis entailed comparing and summarizing survey and interview responses, then contrasting these summaries with the document content analyses. Once this was performed for each case, the cases were compared. Data were analyzed using a priori theory-based codes with deductive coding: assigning labels to data to summarize the basic topic of a qualitative data passage. The coding began with an initial list of codes based upon pre-existing theory, largely from the CHIS database integration studies [32]. Additional codes emerged inductively. All data analyses were performed using ATLAS.ti to code and compare thematic survey responses, interview transcripts and the document contents [33]. A common codebook was established serving as the base for all analyses. Data display matrices were created to illustrate systemic relationships and the within-case and cross-case synthesis. Results Study participants discussed state PHIS database integration projects and the factors that contributed to the projects’ success. Supporting documentation substantiated participants’ claims. Chart 1 illustrates quote frequencies from the survey responses, interview transcripts, and codes from the document review, and it provides a basis for drawing initial conclusions. Chart 1 - Integration Factor Quote Frequency These counts principally illustrate how much the participants spoke about any of the integration project factors, as specified through the coding process. The technological aspects of the Successful Public Health Information System Database Integration Projects: A Qualitative Study Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(2):e207, 2018 7 OJPHI integration projects are discussed most frequently, followed by leadership aspects or cross-cutting departmental projects more generally. Leadership emerges as the most important factor After obtaining information about how the factors contribute to the project’s success, participants were asked which factors were the most important and why. These factors are illustrated in Table 2, which organizes each interview grouped by case, with columns illustrating the most important project integration factors as reported by the interview participants and an overall conclusion from the researcher’s perspective. Leadership aspects of the projects predominate in importance. In particular, participants highlighted the leadership roles of executive sponsors, program directors, and informaticians. Financial Support and Management; Project Management; and the project’s Technology are three other factors that regularly surfaced as important project success factors. Table 2. Reported most important project success factor and explanation Participant by State Most important reported factor(s) Researcher's explanation State B Participant 4 -Financial Support -Organizational Strategy -Technical Support & Coordination Organizational alignment and accreditation set the stage. Executive staff serve as project champions. Informatics business analysts make a difference. Demonstrating value secures flexible funding. Participant 5 -Financial Support -Informatics Leader Dedicated funding is crucial. Informaticists bridge communication gaps. Participants 6 & 7 -Financial Support -Technical Support & Coordination Project completely stalled when the funding temporarily vanished. Well-defined system requirements propel the project. Participant 8 -Leadership, executive* -Financial Support Executive champions play a critical role. Funding is crucial, and can be frustrating. Participant 9 -Leadership, executive* -Financial Support Executive champions and project funding are crucial. State F Participants 19 & 20 -Leadership, program* -Informatician leader Team dynamics and personalities make or break the project. Program-level leadership, not executive support, makes the most difference. Participants 21 & 22 -Leadership, program* -Informatician Leader -Technology Interaction between the tech team and business analyst/informatician is critical. Division-level (not executive) leadership facilitates success. A competent and capable information technology team is key. Successful Public Health Information System Database Integration Projects: A Qualitative Study Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(2):e207, 2018 8 OJPHI -Technical Support & Coordination State A Participant 1 -Organizational Strategy -Informatician Leader -Policy Development Organizational changes linked to strategic planning can have a big impact. Informatician leaders have an enterprise approach. Effective policy facilitates technical decisions. Participants 2 & 3 -Technology -Project Management Source data matters. Dedicate a tech person to the project. Strong project management includes subject matter experts. State E Participant 16 -Leadership, executive* -Informatician Leader Engaged executive leadership provide vision and support, and can facilitate practical changes, such as the shift to Agile project management. Informatics staff lead the projects. Participant 17 -Project Management -Leadership, program* Moving to Agile from Waterfall profoundly impacted the project's success and augmented team synergy. Program directors provide substantive leadership. Participant 18 -Leadership, program* -Financial Support Sustained program director leadership was crucial. Agency timing was right--the will emerged. Federal grants were critical. State G Participant 23 -Leadership, program* Relationships are important. Program director leadership remove obstacles and ensure team synergy. Participant 24 -Leadership, program* -Informatician leader Teamwork is most critical. The involved programs have the same program director and they frequently collaborate. Lead informatician is instrumental in making it a success. State D Participants 14 & 15 -Financial Support -Leadership, executive and program* Federal funding has been critical. The first phase of the projects directly involves senior leadership, whereas latter phases require program leaders to step up. State C Participant 10 -Financial Support -Informatics leader -Leadership, executive* Federal funding for a related initiative was leveraged for this project. Informatician and IT tenacity are critical. Senior-level support and interest are required. Participant 11 -Informatics leader Team dynamics achieve the outcomes. Informatics capacity must be carefully maintained or it can erode. Successful Public Health Information System Database Integration Projects: A Qualitative Study Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(2):e207, 2018 9 OJPHI Participants 12 & 13 -Technology -Leadership, program* Standards makes much of the work possible. Program directors facilitate project success. Funding plays an important prioritization role. Data sharing agreements are necessary. *Illustrates the distinction between executive-level and program-level leadership. Further exploration into the leadership dimension illustrated important nuances. Participants agreed upon the contributions of the informatics staff involved in the initiatives. Informaticians seemed to facilitate the collection of system requirements, translated and communicated needs across project participants and to project sponsors, and developed collaborative team dynamics. However, the contributions of executive leadership and that of program directors were less universally-acknowledged. Some participants attributed project success to the involvement of meaningfully-engaged executive sponsors, whereas others suggested success was due to the regular involvement of program directors directly impacted by the project. Agile emerges as a promising project management technique Project management was indicated by many participants as an important success factor for these initiatives. Table 3 illustrates the project management technique used for each case’s project and a summary of the technique’s impact on the project’s success. All but one case referenced Agile project management in some way, and the participants appeared to express favorable opinions about the role the Agile project management methodology played in the project’s success. Table 3. Project management techniques by case Case Project Management Methodology Project Management impact on project’s success State A Agile with Scrum specifically Scrum techniques facilitate project management. Agency recently moved to Agile model. “In terms of the meetings and stuff go, we are using the Scrum process here, an Agile Scrum process for development, which was also a big change. We used to use Waterfall… But it’s proven that it’s working pretty well, since we switched a couple of years ago.” State B None, although vendor possibly utilized Agile Minimal impact from participants’ perspective. “I think they used the Agile method with short sprints.” State C Agile Agile methodology referenced by one participant but not by others. “We do Agile development. So pretty standardized as far as project management, planning and the reporting is concerned.” State D Agile Regular, sustained activities move the project forward. Successful Public Health Information System Database Integration Projects: A Qualitative Study Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(2):e207, 2018 10 OJPHI “[The NEDSS] uses the agile development approach. All the local users’ representatives really committed a lot of time to do it.” State E Agile Moved to Agile from Waterfall methodology and this change has had a substantive impact, including leading other areas of the agency to adopt the methodology. “We also have moved from a Waterfall method for project management to an Agile methodology. It has made all of the difference in the world; I cannot tell you what a difference it has made. It’s been incredible.” “I think that if we hadn’t had Agile, we still wouldn't have a system up. We would have trashed the build and still wouldn’t have something.” State F Waterfall for most projects but Agile for one Consider Agile to be the better method but not used consistently across the organization. “And Agile to me was superior and definitely what we should implement here.” State G None Minimal impact from participants’ perspective. The participants described that by utilizing Agile, the program staff, informatics personnel, technological developers, and others involved in the project closely collaborated in the development of the integrated database, producing a better product and overall experience than that through traditional software development techniques. One case’s participants in particular, State E, stated that the shift to Agile from the Waterfall methodology introduced a profound cultural shift within the agency that transcended PHIS database integration and general information technology project management. Other areas of the agency began applying Agile methodology to other projects based upon the success identified in its use with the PHIS database integration projects. Participant 16 described this profound shift: “Since that time we don’t do anything but Agile. What’s really cool is some of our business side—our service areas—want to start using Agile with their staff, because it holds people accountable. You have to stand up and say ‘This is what I did yesterday, and this is what I’m going to do today.’ Everybody gets to hear it; everybody has to be accountable. It builds that team that you just don’t always see with things like that. It was a profound difference, I’ll tell ya.” This shift to Agile methodology had substantively changed the project management experience for some of the study’s participants. Participant 17 from State E suggested that the utilization of the Agile methodology was the most important success factor for the project: “I definitely think it’s the Agile process in and of itself. It helped the project move forward. Even when we had a roadblock it’s not like everything just stopped…It created this wonderful team atmosphere where everybody knows we’re working for this same end goal.“ Successful Public Health Information System Database Integration Projects: A Qualitative Study Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(2):e207, 2018 11 OJPHI “Switching to Agile made a huge difference, and I would recommend it for any process.” “So it’s a very interactive, engaged process. It’s incredible, I’ll tell ya. I’ve been amazed at the differences—-the speed at which a project gets done. All of that front-end time is lost.” Agile clearly changed the way the health department conducted business, and positively impacted many of the integration projects described in this study. Consistency across cases The case summary table (Table 4) illustrates cross-case comparisons and distinct features of each. The participants’ remarks from each case seem to consistently reflect across the spectrum of cases. Funding is cited as a project catalyst, and Leadership involvement across multiple levels of the organization ensures project success in various ways. Technological factors such as the quality and structure of source data, ensuring standards are employed, and maintaining data warehousing expertise impact the development of the integrated database. Effective project management facilitates project success, and Agile is regularly referenced as a useful method. However, important differences surface when participants describe the contributions of the executive leaders compared with those of program directors. As alluded to in Table 2, some cases evenly highlight the contributions of both groups, whereas other cases are characterized by substantive involvement of either executive leaders or program directors, but not both. Table 4. Case Summary Table State Summary State A Strategic planning and policy development set the project context. The Informatician plays a critical role by fully engaging team members. The quality of the source data impacts development progress. State B Executive leaders align resources and seed funding. Informaticians collect thorough business requirements. Prior strategic planning that addressed information management principles seems to have had a lasting positive effect. State C Leadership by the program staff and informatician ensure functional team dynamics. Technological standards facilitate other programs’ integration efforts. Executive support and interest bolster project activities. Funding is critical. State D Executive leadership set the project vision and initial activities, and strong program and bureau leaders are required for project sustainment. Funding is essential. State E Program and executive-level leadership both impact the project. An agency-wide shift to Agile project management changed the organizational culture and facilitated success. Informatics staff lead these initiatives. Funding was crucial. Successful Public Health Information System Database Integration Projects: A Qualitative Study Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(2):e207, 2018 12 OJPHI State F Program-level leadership and informatics leaders promote healthy relationships and team dynamics. Information Technology team member permanence ensures continuity. State G Program directors and informaticians ensure success by fostering functional team dynamics and relationships. Discussion Principal Findings The study’s principal findings illustrate the complex involvement in PHIS database integration project leadership beyond the role of the executive leaders. Much is known about how executive leaders contribute to project success through their sponsorship and support. This study suggests that other project participants, notably informaticians and program directors, substantially contribute to the project’s success. While executive involvement might be critical for initiating the project, program directors ensure project staff remain engaged, and informaticians provide a crucial role in facilitating project conversations across diverse participants. Additionally, the Agile software development methodology is emerging as a successful approach to project management for these and related projects. Some participants claim adopting this approach introduced a dramatic shift in how the integration projects progressed, and one suggested this was the main reason that site’s project succeeded. Agile improves project accountability and team member involvement and interaction, while speeding the deployment of useful software. Implications This study has three primary implications. First, developing the leadership skills of informaticians, relevant program directors, and executive leaders may promote the success of these and related initiatives. Since these projects require informatics savviness, these individuals may benefit from informatics training more generally, and PHIS database integration training specifically. Secondly, project financing challenges are not new to public health departments, and this aspect seems to impact PHIS database integration project success, especially the launching of these initiatives. Federal programs have funded these efforts in the past, and future funding could facilitate their initiation. Finally, employing formal project management techniques might ensure the project runs smoothly. Investing in Agile methodology training and enabling its use could be an effective approach to ensuring the project is properly managed. Limitations This study has three principal limitations. First, cases purposefully recruited represent an exemplary and small subset of all state public health departments. Therefore, the study’s results should not be interpreted as representative of all state health departments. Secondly, the data are based on survey responses, interviews, and a document review. Participant responses may be affected by subjectivity, and undiscovered documentation may suggest alternative conclusions. The data had not been triangulated with onsite visits and additional observations to corroborate findings. Thirdly, a single researcher performed the data collection, coding, and analysis. Inclusion Successful Public Health Information System Database Integration Projects: A Qualitative Study Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(2):e207, 2018 13 OJPHI of another researcher could confirm codes and findings. Despite these limitations, the study’s findings provide useful insight into integration project success. More research in this area is needed to further understand this topic. Conclusion This study improves the understanding of the most important public health information system database integration project success factors. Public health database integration needs persist, and stakeholders may use these findings to improve the likelihood of future project success. Acknowledgments I would like to acknowledge the following individuals: my committee chair and advisor, Dr. Patrick Lenihan, and committee members Dr. Eve Pinsker, Dr. Seth Foldy, Dr. Edward Mensah, and Dr. Kee Chan. I also thank the study’s participants for their insight and wisdom. Financial Disclosure No Financial Disclosures Competing Interests No Competing Interests References 1. Ammenwerth E, Graber S, Herrmann G, Bürkle T, König J. 2003. Evaluation of health information systems—problems and challenges. Int J Med Inform. 71, 125-35. PubMed https://doi.org/10.1016/S1386-5056(03)00131-X 2. Yasnoff WA, O’Carroll PW, Koo D, Linkins RW, Kilbourne EM. 2000. Public Health Informatics: Improving and transforming public eHealth in the information age. J Public Health Manag Pract. 6(6), 67-75. PubMed https://doi.org/10.1097/00124784-200006060-00010 3. Magruder C, Burke M, Hann NE, Ludovic JA. 2005. Using Information Technology to Improve the Public Health System. J Public Health Manag Pract. 11(2), 123-30. PubMed https://doi.org/10.1097/00124784-200503000-00005 4. Arzt NH. Evolution of Public Health Information Systems: Enterprise-Wide Approaches. San Diego: HLN Consulting; 2007. 5. Mannino M, Hong SN, Choi IJ. 2008. Efficiency evaluation of data warehouse operations. Decis Support Syst. 44, 883-98. https://doi.org/10.1016/j.dss.2007.10.011 6. Foldy S, Grannis S, Ross D, Smith T. 2014. A Ride in the Time Machine: Information Management Capabilities Health Departments Will Need. Am J Public Health. 104(9), 1592-600. PubMed https://doi.org/10.2105/AJPH.2014.301956 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=14519405&dopt=Abstract https://doi.org/10.1016/S1386-5056(03)00131-X https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=18019962&dopt=Abstract https://doi.org/10.1097/00124784-200006060-00010 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=15711442&dopt=Abstract https://doi.org/10.1097/00124784-200503000-00005 https://doi.org/10.1016/j.dss.2007.10.011 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=25033122&dopt=Abstract https://doi.org/10.2105/AJPH.2014.301956 Successful Public Health Information System Database Integration Projects: A Qualitative Study Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(2):e207, 2018 14 OJPHI 7. Lumpkin J, Atkinson D, Biery R, Cundiff D, McGlothlin M, et al. 1995. The development of integrated public health information systems: a statement by the Joint Council of Governmental Public Health Agencies. J Public Health Manag Pract. 1(4), 55-59. PubMed 8 Linzer DS, Lloyd-Puryear MA, Mann M, Kogan MD. 2004. Evolution of a Child Health Profile Initiative. J Public Health Manag Pract. November(Suppl):S16-S23. 9 Hinman AR, Delton A, Diehn TN, Eichwald J, Heberer J, Hoyle T, King P, Kossack RE, Williams DC, Zimmerman A. 2004. Principles and Core Functions of Integrated Child Health Information Systems. J Public Health Manag Pract. November (Suppl):S52-S56. 10 Saarlas KN, Hinman AR, Ross DA, Watson WC Jr, Wild EL, Hastings TM, Richmond PA. 2004. All Kids Count 1991–2004: Developing Information Systems to Improve Child Health and the Delivery of Immunizations and Preventive Services. J Public Health Manag Pract. November(Suppl),S3–S15. 11. Davenport DH. Mission Critical: Realizing the Promise of Enterprise Systems. Boston, Massachusetts: Harvard Business School Press; 2000. 12. Hasbrouck L. 2016. Strengthening Local Health Department Informatics Capacity Through Advocacy, Education, and Workforce Development. J Public Health Manag Pract. 22(6) (Supp), S3-5. PubMed https://doi.org/10.1097/PHH.0000000000000475 13 Wild EL, Fehrenbach SN. 2004. Assessing Organizational Readiness and Capacity for Developing an Integrated Child Health Information System. J Public Health Manag Pract. November(Suppl):S48-S51. 14 Fehrenbach SN, Kelly JCR, Vu C. 2004. Integration of Child Health Information Systems: Current State and Local Health Department Efforts. J Public Health Manag Pract. November(Suppl):S30-S35. 15. Hwang HG, Ku CY, Yen DC, Cheng CC. 2004. Critical factors influencing the adoption of data warehouse technology: a study of the banking industry in Taiwan. Decis Support Syst. 37, 1-21. https://doi.org/10.1016/S0167-9236(02)00191-4 16. Wixom BH, Watson HJ. 2001. An Empirical Investigation of the Factors Affecting Data Warehousing Success. Manage Inf Syst Q. 25(1), 17-41. https://doi.org/10.2307/3250957 17. Joshi K, Curtis M. 1999. Issues in building a successful data warehouse. Information Strategy: The Executive’s Journal. Military & Government Collection, Ipswich, MA. 15(2), 28. Accessed July 19, 2015. 18. Watson HJ, Fuller C, Ariyachandra T. 2004. Data warehouse governance: best practices at Blue Cross and Blue Shield of North Carolina. Decis Support Syst. 38, 435-50. https://doi.org/10.1016/j.dss.2003.06.001 19. Ramamurthy KR, Sen A, Sinha AP. 2008. An empirical investigation of the key determinants of data warehouse adoption. Decis Support Syst. 44, 817-41. https://doi.org/10.1016/j.dss.2007.10.006 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=10186643&dopt=Abstract https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=27684615&dopt=Abstract https://doi.org/10.1097/PHH.0000000000000475 https://doi.org/10.1016/S0167-9236(02)00191-4 https://doi.org/10.2307/3250957 https://doi.org/10.1016/j.dss.2003.06.001 https://doi.org/10.1016/j.dss.2007.10.006 Successful Public Health Information System Database Integration Projects: A Qualitative Study Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(2):e207, 2018 15 OJPHI 20. Markus ML. 1983. Power, politics, and MIS implementation. Commun ACM. 26(6), 430-44. https://doi.org/10.1145/358141.358148 21. Rizi S, Roudsari A. 2013. Development of a public health reporting data warehouse: lessons learned. Medinfo. •••, 861-65. PubMed 22. Boehm BW. 1988. A spiral model of software development and enhancement. Computer. 21(5), 61-72. https://doi.org/10.1109/2.59 23. Meyer B. Agile!: The Good, the Hype and the Ugly. Zurich, Switzerland: Springer International Publishing Switzerland; 2014. 24. Raghu A, Praveen D, Peiris D, Tarassenko L, Clifford G. 2015. Engineering a mobile health tool for resource-poor settings to assess and manage cardiovascular disease risk: SMARThealth study. BMC Med Inform Decis Mak. 15, 36. PubMed https://doi.org/10.1186/s12911-015-0148-4 25. Kitzmiller R, Hunt E, Sproat SB. 2006. “Agility” Moves From Software Development to Healthcare Project Management. Comput Inform Nurs. 24(2), 75-82. PubMed https://doi.org/10.1097/00024665-200603000-00005 26. Kane DW, Hohman MM, Cerami EG, McCormick M, Kuhlmman KF, et al. 2006. Agile methods in biomedical software development: a multi-site experience report. BMC Bioinformatics. 7, 273. PubMed https://doi.org/10.1186/1471-2105-7-273 27. Christy A. 2016. Government Goes Agile. Stanf Soc Innov Rev. (Spring), 13-14. 28. Misra SC, Kumar V, Kumar U. 2009. Identifying some important success factors in adopting agile software development practices. J Syst Softw. 82, 1869-90. https://doi.org/10.1016/j.jss.2009.05.052 29. Yin RK. Case study research: design and methods—4th ed. Thousand Oaks, California: Sage Publications, Inc.; 2009. 30. Qualters JR, Strosnider HM, Bell R. 2015. Data to Action: Using Environmental Public Health Tracking to Inform Decision Making. J Public Health Manag Pract. 21(2) (Supp), S12-22. PubMed https://doi.org/10.1097/PHH.0000000000000175 31. Bara D, McPhillips-Tangum C, Wild EL, Mann MY. 2009. Integrating Child Health Information Systems in Public Health Agencies. J Public Health Manag Pract. 15(6), 451-58. PubMed https://doi.org/10.1097/PHH.0b013e3181abbec8 32. Maxwell JA. Qualitative research design: an interactive approach-3rd ed. Thousand Oaks, California: Sage Publications, Inc.; 2013. 33. Miles MB, Huberman AM, Saldaña J. Qualitative data analysis: a methods sourcebook—3rd ed. Thousand Oaks, California: Sage Publications, Inc.; 2014. https://doi.org/10.1145/358141.358148 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23920680&dopt=Abstract https://doi.org/10.1109/2.59 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=25924825&dopt=Abstract https://doi.org/10.1186/s12911-015-0148-4 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=16554690&dopt=Abstract https://doi.org/10.1097/00024665-200603000-00005 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=16734914&dopt=Abstract https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=16734914&dopt=Abstract https://doi.org/10.1186/1471-2105-7-273 https://doi.org/10.1016/j.jss.2009.05.052 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=25621441&dopt=Abstract https://doi.org/10.1097/PHH.0000000000000175 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=19823148&dopt=Abstract https://doi.org/10.1097/PHH.0b013e3181abbec8 Successful Public Health Information System Database Integration Projects: A Qualitative Study Abstract Introduction Methods Survey and interview questions Procedure Analyses Results Leadership emerges as the most important factor Agile emerges as a promising project management technique Consistency across cases Discussion Principal Findings Implications Limitations Conclusion Acknowledgments Financial Disclosure Competing Interests References