Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Aliza Monroe-Wise1, John Kinuthia2, Sherrilynne Fuller3, Matthew Dunbar4, David Masuda5, Elisha Opiyo6, Betty Muchai1, Christopher Chepken6, Elijah Omwenga6, Robert Oboko6, Alfred Osoti2, Daniel Masys5, Michael H. Chung 1 1Departments of Global Health and Medicine, University of Washington 2Department of Obstetrics and Gynaecology, University of Nairobi, Nairobi, Kenya 3Department of Biomedical Informatics and Information School, University of Washington 4Center for Studies in Demography and Ecology, University of Washington 5Department of Biomedical Informatics and Medical Education, University of Washington 6School of Computing and Informatics, University of Nairobi Abstract Objectives Information and communication technology (ICT) tools are increasingly important for clinical care and international research. Many technologies would be particularly useful for healthcare workers in resource-limited settings; however, these individuals are the least likely to utilize ICT tools due to lack of knowledge and skills necessary to use them. Our program aimed to train researchers in low-resource settings on using ICT tools and to understand how different didactic modalities build knowledge and skills in this area. Methods We conducted a tiered, blended learning program for researchers in Kenya on three areas of ICT: geographic information systems, data management, and communication tools. Each course included three tiers: online courses, skills workshops, and mentored projects. Concurrently, a training of trainers course was taught to ensure sustainable ongoing training. A mixed qualitative and quantitative survey was conducted at the end of each training to assess knowledge and skill acquisition. Results Course elements that incorporated local examples and hands-on skill building activities were most valuable. Discussion boards were sometimes distracting, depending on multiple factors. Mentored projects were most useful when there were clear expectations, pre-existing projects, and clear timelines. Discussion Training in the use of ICT tools is highly valued among researchers in low-income settings, particularly when it includes hands-on skill-building and local examples. Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI 1. Introduction Information and Communication Technology (ICT) tools have transformed the ways in which research and healthcare are conducted and data is managed, becoming integral components of both biomedical research and healthcare delivery throughout the world [1-3]. These technologies can change the ways in which medical data is collected and managed for clinical trials by using mobile devices [4], providers communicate with patients for monitoring health and disease [5], and the creation of large interoperable data management systems for analysis across multiple platforms and studies [6]. Many of these ICT tools are accessible and freely available on the internet or as open source software [3,7]. While innovative ICT practices would have the highest impact on health research and patient care in resource-limited settings [8], these areas often have the lowest ICT uptake and utilization rates of anywhere in the world [9,10]. Lack of knowledge and skills have been identified as primary barriers to the use of ICT tools among health researchers in resource-limited settings [10-12], and there is an increasingly recognized need for improved access to ICT training opportunities for health researchers worldwide. It is clear that international collaboration is vitally important for building research capacity in resource limited academic centers [13], and also clear that ICT tools can improve the efficiency of multinational research projects that span cultures, languages and time zones [14,15]. As such, international collaborations may both facilitate training surrounding the use and integration of ICT tools and also support the research resulting from increased utilization of the tools [16,17]. In Kenya, while certain research projects have successfully been implemented in recent years using select ICT tools [18,19], a great need exists to increase knowledge and understanding of new technologies in order to expand utilization [20,21], similar to other resource-limited settings. In response to the identified need for improved and expanded training programs in ICT utilization for Our students demonstrated acquisition of new skills and felt these skills to be valuable in their workplaces. Conclusions Further training in ICT skills for researchers should be considered in other low-resource settings using our program as a foundational model.Key Words: Information and communication technology (ICT), blended learning, Kenya, e-learning Corresponding author: Aliza Monroe-Wise, MD, MSc. Email: alizamw@uw.edu University of Washington, 325 Ninth Avenue, Box 359909, Seattle, WA 98104-2499 DOI: 10.5210/ojphi.v11i3.10323 Copyright ©2019 the author(s) This is an Open Access article. Authors own copyright of their articles appearing in the Online Journal of Public Health Informatics. Readers may copy articles without permission of the copyright owner(s), as long as the author and OJPHI are acknowledged in the copy and the copy is used for educational, not-for-profit purposes. Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI health research in Kenya, the University of Washington (UW) partnered with the University of Nairobi (UoN) to develop a training program to meet the needs of local researchers in Nairobi. We also aimed to gain a deeper understanding of which modalities of ICT training were most and least valuable to the facilitation of applied learning in this setting through post-training survey data collection. This paper describes the design and content of our training program, and the evaluation of post-training survey data describing trainees’ views on how various training components fostered the development of ICT knowledge and skills. 2. Materials and Methods Leading faculty from UW and UoN in the areas of computing, biomedical informatics, geographic information systems, and e-Learning convened in January 2015 to collaboratively identify target areas for training and appropriate and effective teaching methods to deliver content. They drew on existing research on the use of ICT in biomedical research, experience with effective delivery platforms, and adult learning theory to inform the development of both course content and delivery, and they took several factors into consideration. First, given the importance of international collaboration, the inclusion of international faculty as lecturers in the program was prioritized. Given increasing evidence supporting the efficacy of e-Learning modalities for distance training [22,23], an e-Learning component was included in order to deliver content from international experts in a financially sustainable manner. Finally, faculty identified a need to include skills transfer and mastery, in addition to knowledge delivery. Studies have shown that blended learning, in which online content is combined with in-person training, is most effective for educational programs that focus on the development of skills [24]; therefore a blended learning approach was taken with an online course featuring international experts followed by mentored, in-person learning. The faculty designed a three-tiered, blended learning program (Figure 1). Three separate tracks of training were chosen to correspond to areas of highest need and interest: Geographic Information Systems (GIS), Principles and Practice of Research Data Management (PPR), and Research Management and Communication Tools (RCT). For each of these training tracks, three tiers of training were offered. The first tier was an online course consisting of a series of 4-6 lectures (in voice over PowerPoint format), in conjunction with other educational materials, homework and assignments. Tier 1 target enrollment was 100 students per course. Tier 2 consisted of hands-on workshops for each topic that took place over 5 days after the completion of online courses. Participants were selected from those who had completed and passed the online course for each topic. Tier 2 workshop target enrollment was 30 students per workshop. Finally, the top 15 students from each workshop were identified based on scores from final presentations. These top 15 were then invited to participate in Tier 3 for each topic, intensive mentored projects. Mentors were chosen from University of Nairobi ICT experts in each topic, and the mentorship experience lasted 3 months. Mentors met with students regularly and guided students through the application of an ICT tool to a research project. Students were tasked with presenting their projects via either an oral presentation online or at the University of Nairobi STD/AIDS Collaborative Group Conference held in Nairobi annually. The entire 3-tier training took place over a 6-month timeframe (Figure 2). Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI Figure 1. Overall training structure Figure 2. Training timeframe A training of trainers (TOT) course was simultaneously offered to all faculty and staff at UoN who provide ICT support and training. This course was led by ICT specialists from both UW and UoN, and comprised an initial introductory online course that also served as an introduction for the other courses followed by a 5-day hands-on workshop. Participants learned how to: help researchers improve their research design and analysis using ICT; mentor researchers constructively in their projects; and teach researchers to give effective oral presentations. Post-course surveys were offered to participants in each element of this training program. Surveys were conducted electronically for online courses, and on paper for in-person workshops and mentoring. Surveys comprised both quantitative and qualitative questions covering topics about length of time participants took for the training program, which elements of the course or program were most valuable and least valuable, and whether and how the participants’ knowledge and skills were improved from what they learned in the course. Surveys also assessed whether participants felt they could use the knowledge and skills acquired in their workplaces. Programmatic data was also collected in the evaluation, including data on course enrollments, course completion, pass rates for each course, and deliverables expected and produced for each course as supporting information to describe skill acquisition. Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI 2.1 Course Content 2.1.1 Introduction to ICT in Health Research (INTRO) and Training of Trainers (TOT) The goal of the intro course was twofold: first, to provide foundational knowledge to a cadre of faculty and staff who could ensure sustainability and extensibility of the training program and as a resource for ICT applications in health research; second, to provide a common introduction to concepts for students in the other three courses. Detailed information about objectives and activities for all four courses can be found in Appendix A. The core online intro course considered the state of the art data management tools for ensuring reliable collection, aggregation, back-up, analysis, reporting and archiving of data in useful ways. A unique focus of the course was assessing the ability of tools to work together to successfully ensure smooth movement of data and findings from one tool to the next. Another focus of the course was evaluation of health research studies to identify those which successfully (or unsuccessfully – thus covering the research pitfalls) incorporate ICT in resource-limited settings. Because ICT changes so rapidly, the course also covered how to remain informed of emerging ICT approaches and also how to successfully plan for adaptation to future technology changes. For faculty and staff already familiar with ICT, the course provided an introduction to the needs of health researchers and the issues that they face in conducting clinical trials or medical investigations research. The use and research applications of social technologies including crowdsourcing and social media was a particular focus of the course. The online course was followed by a five-day in-person workshop for faculty in the training the trainers (TOT) course. The workshop allowed participants to apply their learning from the online course in a hands-on laboratory setting, and provided an opportunity for ICT experts and health researchers to meet each other and teach each other the reciprocal skills of ICT and health research as well as to collaborate to design new solutions to research challenges. Teaching assistants for the other 3 courses were chosen from this TOT course. 2.1.2 Geographic Information Systems (GIS) The foundation of this course introduced participants to key features of the discipline of Geography, and explored how geospatial technologies, such as GIS, are commonly used to incorporate spatial theory, analysis, and visualization into health research. The remaining majority of the course was focused on the following objectives: learning how to assess the use of GIS in research, performing basic mapping and analysis using open-source GIS software, and developing and implementing spatial research questions using GIS. This course attempted to teach both the theory to understand how GIS is used in research (weekly lectures), and the technical software skills necessary to implement a subset of those concepts (weekly assignments). The workshop component of the GIS course focused on expanding the practical technical skills of the participants, while concurrently deepening their understand of geographic science. Four workshop modules (Mapping, Analysis, Data Manipulation, and Rasters) provided practical training to advance the participants competence in QGIS from the introductory level in the online course to an intermediate skill level. Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI 2.1.3 Principles and Practice of Research Data Management (PPR) The PPR course was anchored by three conceptual areas that are converging worldwide in research that involves human volunteer participants. The first of these are the international Good Clinical Practice (GCP) standards developed by the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use [25]. GCP standards have become the accepted global guidelines for ensuring ethical treatment of research participants and creation of research records that are of high quality and amenable to audit for scientific integrity. The second theme of the PPR course was knowledge of data modeling, database design, and data management technologies capable of supporting research ranging from simple single investigator studies to international multisite clinical trials. The third theme was understanding the strengths and hazards of using the internet to acquire and manage research data, with a focus on principles and practice of information security. The independent study assignment for the course required students to bring these related sets of knowledge and skills together to create a research data management plan, including budget, that could become part of a research study proposal to a sponsoring organization. 2.1.4 Research Management and Communication Tools (RCT) The course was informed by the centrality of the internet as a tool in the research process enabling researchers to access funding, identify collaborative partners, keep abreast with advances in research, and disseminate their research findings quickly and effectively. Skills-based learning objectives are outlined in Table 1. The course focused on open access ICT tools that researchers can creatively leverage in tandem to better manage the different stages of the research process, from proposal writing through to research dissemination and uptake. A key focus of the course was hands-on practice on various research indexing tools, bibliography management tools, research networking tools and platforms, online collaborative tools, social media platforms, data sharing tools and platforms, data back-up and archival tools and platforms, presentation tools, and low-cost communication tools. Effective use of a variety of search engines to enhance retrieval of relevant research findings was threaded throughout the course. Students benefitted greatly from demonstrations of tools and technologies by their peers. Finally, the course equipped learners with strategies for creating an effective, online personal brand to enhance their global visibility, extend their research dissemination audience, and increase the uptake potential of their research. 3.Course Evaluation Results 3.1 Overall Participation Rates & General Feedback A total of 978 individuals applied to take the four online courses (Table 2). Acceptance into courses was based on educational level (high school certificate or higher) and country of residence. Pass rates ranged from 42% for the RCT course to 100% for the TOT course and workshops. The low pass rate for the RCT course was due to the required discussion board participation and difficulty level of the material. Finally, the 27 students who participated in three-month mentored projects were tasked with producing a deliverable at the end of the training period. Overall, 23 of the 27 Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI enrolled successfully submitted deliverables, producing a pass rate of 85% for the mentored projects. Online courses and workshops were generally well-received in terms of the timing and pace, organization, and overall effectiveness of the courses. The majority of learners spent between 2 and 6 hours per week on online courses. Students occasionally encountered technical challenges, particularly during the online courses. About 15% of respondents in both the GIS and RCT online courses reported having some technical difficulties accessing materials, the most common of which were due to slow internet limiting access to lectures or difficulty logging in to course content. The most commonly cited elements that posed a distraction to learning in the online courses were discussion boards and technical difficulties. Other occasionally cited distractions included problems with learning the content, format of the content (i.e the way in which content was presented), and having too much ancillary content presented in the form of links or additional readings. Table 2. Acceptance, completion and pass rates for each course Training of Trainers (TOT) Geographic Information Systems (GIS) Principles & Practice of Research Data Management (PPR) Research Management & Communication Tools (RCT) Totals Online Course Total applicants 47 150 397 384 978 Participants1 34 (72%) 132 (88%) 241 (61%) 319 (83%) 726 (74%) Completed2 32 (94%) 107 (81%) 188 (78%) 232 (73%) 559 (77%) Passed3 32 (100%) 84 (78%) 155 (82%) 98 (42%) 369 (66%) Workshops Total applicants 32 61 87 41 221 Participants1 32 (100%) 28 (46%) 30 (34%) 29 (71%) 119 (54%) Completed2 32 (100%) 28 (100%) 30 (100%) 29 (100%) 119 (100%) Mentored Projects Participants N/A 10 7 10 27 Completed 6 (60%) 7 (100%) 10 (100%) 23 (85%) 1Percentage accepted for each course out of total applicants Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI 2Percentage who completed course assignments out of total number registered 3Percentage who passed the course out of total who completed Figure 4. Valuable components of online courses Figure 5. Valuable components of workshops 3.3 Online Courses While online courses generally received high scores in course evaluations, the most valuable components of these courses varied greatly between the different tracks (Figure 4). Lectures were considered greatly valuable for the Training of Trainers (TOT) course, whereas assignments were considered the most valuable component of the Geographic Information Systems (GIS) course. In general, online courses with active learning components (GIS and PPR) were considered especially Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI valuable. GIS included weekly activity assignments in which learners were tasked with locating and utilizing mapping resources available on the internet to design, interpret or modify maps. These exercises were widely lauded as being extremely valuable to the learning process. Similarly, the PPR course included an independent project which was one of the highlights of the course. Conversely, the RCT course did not include any practical component, and this was noted by several respondents in the course evaluations with comments such as “increase practical application,” and include “guided practice of the tools.” With the lowest pass rate of the online courses, the RCT course was notable in its use of mandatory discussion board assignments rather than active learning components, which may have hindered learning efficacy. One of the well-received assignments in the PPR course utilized published clinical research studies from Kenyan researchers to understand the research methods and review the ICT methods and tools that were used in the study as well as to identify what additional ICT tools and methods might have been used. Linking actual Kenyan research studies to use of ICT tools and technologies proved very effective as a practical teaching aid. Similarly, throughout the PPR course, the students successfully partnered with others who had complementary skills, pairing those who had experience in research methods and ICT with those who did not. Those with clinical research experience could give examples of their own research experience and challenges as it related to the use of specific ICT tools they were learning about. Discussion boards received mixed reviews from students in the different courses. Some felt that the discussion boards were quite a valuable addition to the other components of the course, while others ranked discussion boards as the most distracting feature of the course. Supervision and structure in discussion board interactions added more value to this component of the different courses. 3.4 Workshops Workshops were very well received, earning higher average overall scores than either the online courses or the mentored projects. Similar to online courses, the most valuable elements of each workshop differed greatly (Figure 5). While practical skills acquired during workshops were among the most highly valued element by participants in the GIS and RCT workshops, the interactive nature of the TOT workshop was its most valued component and the active learning assignments were the highlight of the PPR workshop. Lectures and theory were the lowest rated component of all four workshops. Specific skills learned that were mentioned frequently were working with shape and raster files, accessing online data and utilizing data for map creation, using plug-ins for spatial analysis, and vector analysis in GIS; how to collect high quality data, maintain data accuracy, set up and configure servers, ensure data security using encryption, and programming ODK and Redcap in PPR; and mind maps, data sharing and management tools, project management and collaboration tools, conceptual framework formulation and referencing tools in RCT. Workshops also increased exposure to tools the students had never before heard of, broadening their understanding of the vast array of ICT applications available. Students largely felt that the workshops should have been longer, as there were additional skills and tools they had hoped to master. One limitation of the Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI workshops mentioned by several participants was that Android devices were not provided, but were necessary for some of the exercises. 3.5 Mentored Projects Overall, mentored projects received mixed reviews with lower average scores than the other two components of the training. While the value of the mentorship experience was felt to be immense when mentorship was carried out in an ideal manner, there were many obstacles to successful engagement in mentored projects. A common pitfall described in all three courses was lack of clarity in expectations for the program and for completion of the project, with around 40-50% of students stating that expectations were clear in both PPR and GIS. Each mentee was required to produce a deliverable by the end of the mentorship period. In the GIS course, students submitted abstracts with maps that showcased their use of GIS concepts and technology. For PPR, mentees sent evidence of forms they had designed to collect data, in the form of either screenshots or downloadable ODK forms. In the RCT course mentees demonstrated the use of three different tools that were discussed in the workshop by submitting screenshots of engagement with the tools. Despite these goals, mentees did not feel well informed regarding mentored project expectations. In the GIS mentored projects, the lack of clarity on project deliverables may have contributed to the low pass rate of 60%. Having timelines for project completion was mentioned as one way to improve expectations in several of the courses. Another frequently cited obstacle was difficulty engaging with mentors, due to lack of mentor’s available time or failure to establish a clear schedule for meetings. One participant recommended that mentor-mentee meetings be mandatory at regular intervals to ensure adequate meeting frequency. Another suggestion was to include meetings with other trainees to add a peer mentoring component. Finally, similarly to the workshops, some participants mentioned that it was problematic when students were required to produce their own project ideas with supporting data, as several of the students did not have data readily available and suggested that the mentors have data for use in the mentorship projects. Overall when the mentored projects went well, they were considered extremely useful. One participant in the RCT mentorship project raved “This was a wonderful opportunity that changed my academic life, research work, and approaches for the better of my future career in research.” 4. Discussion While information and communication technology (ICT) has become integral to the successful implementation of research and health delivery worldwide, there remains a gap in the utilization of these tools among researchers in resource-limited settings, largely due to the lack of education and training programs available. Drawing on an international collaboration to expand expertise and reinforce the importance of global collaborations in research, we designed and conducted a group of tiered, blended learning courses focused on training Kenyan researchers in the use of ICT tools. We collected data from our post-course evaluation surveys to better understand how different didactic modalities functioned to deliver both content and skills in ICT. Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI Previous programs aimed at providing ICT training to audiences in low-resource settings are limited, but highlight the importance of relevance of the concepts taught to the participants’ work [26]. Our program affirmed the need for relevance, with highly rated activities being those that incorporated local examples or skills that could be easily applied to everyday work. Many of the most highly valued content elements taught in all of the courses were those that focused on how to use specific tools, such as RedCap, ODK, Mendeley, and Google tools. We found the tiered training program to be useful, although both the online course and the mentorship components were subject to pitfalls. Technical and infrastructure challenges are common barriers to implementing e-Learning programs [27,28] but might be overcome through stakeholder and institutional support [27]. Although several studies have highlighted the utility of discussion boards [29,30] and they are generally considered to be productive components of online learning [31], they have been noted to be problematic before [30,32], largely due to lack of perceived need and unfamiliarity with the technology. Our program data demonstrated that discussion boards were most useful when provided in a structured manner, with instructors regularly logging in and monitoring the interactions taking place, echoing previous research showing that the type and frequency of instructor interaction in online discussion boards is vital to their didactic success [31]. While mentorship has been shown to positively affect mentees’ career choices and development [33], pitfalls can be common [34,35]. Mentored projects in our training were subject to many of these obstacles, and were most useful when projects and data were available at the start of the project, mentors had sufficient time to provide considerable effort and oversight, and when expectations, timelines and deadlines were made clear at the beginning of the project period. Finally, previous studies have emphasized the workshop format as an ideal modality for delivering skills-based training in low-resource settings, particularly in the realm of ICT [36]. Our evaluation results underscore this finding, with the workshops succeeding in delivering both knowledge and skills without the obstacles that are commonly encountered in e-Learning formats and during mentorship. The interpretation of data from this study is subject to several potential limitations. First, our surveys all relied on self-reported measures, which can be influenced by desirability and recall biases, among others. We mitigated these issues by conducting surveys immediately following trainings, and by using normalizing language where possible. Additionally, we have triangulated self-reported data with more objective measures, such as pass rates and deliverables produced for each course. Second, outcomes from our training programs may not be applicable to other low- resource settings in Africa or elsewhere. While trainings must be tailored to their specific audience and setting, we believe that the core concepts and structure of our ICT courses can be applied broadly. As training in ICT becomes more important for researchers worldwide, the need for effective educational modalities to provide knowledge and skills to researchers in resource-limited settings is increasingly apparent. Our training program may serve as an excellent foundation on which future courses in ICT may be based. Additionally, the inclusion of the TOT course contributes significantly to the didactic sustainability of the course content, and may become the basis of similar, locally-offered courses in the future. Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI Authors’ Contributions MC, JK and EO conceived of the project. MC, JK, EO, BM, SF, MD, DM, CC, EO, RO, AO and DM implemented the training or designed and taught courses in the training program. AMW, MC, BM and JK analyzed the data. AMW, DM, SF and MC wrote the manuscript and all authors reviewed the manuscript for content. Acknowledgements The authors wish to thank the students and faculty at the University of Nairobi for their participation in this training activity. Funding This training was funded by NIH grant 1R25TW009692. The funding agency did not play a role in the research design, data collection, data analysis or interpretation. Ethical Approval The collection of data for this study was approved the Kenyatta National Hospital Ethics and Research Committee and was deemed exempt by the University of Washington Division of Human Subjects. Competing Interests The authors state that they have no competing interests. References 1. Friedman DJ, Parrish RG, Ross DA. 2013. Electronic Health Records and US Public Health: Current Realities and Future Promise. Am J Public Health Wash. 103(9), 1560-67. PubMed https://doi.org/10.2105/AJPH.2013.301220 2. Institute of Medicine. Digital Infrastructure Workshop Highlights [Internet]. 2011 [cited 2018 Aug 16]. Available from: http://www.nationalacademies.org.offcampus.lib.washington.edu/hmd/~/media/Files/Report %20Files/2010/Digital-Infrastructure-for-the-Learning-Health- System/Digital%20Infrastructure%20Workshop%20Highlights.pdf 3. Fuller S. 2010. Tracking the Global Express: New Tools Addressing Disease Threats Across the World. Epidemiology. 21(6), 769-71. PubMed https://doi.org/10.1097/EDE.0b013e3181f56757 4. Zhang J, Sun L, Liu Y, Wang H, Sun N, et al. 2017. Mobile Device-Based Electronic Data Capture System Used in a Clinical Randomized Controlled Trial: Advantages and Challenges. J Med Internet Res. 19(3), e66. PubMed https://doi.org/10.2196/jmir.6978 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23865646&dopt=Abstract https://doi.org/10.2105/AJPH.2013.301220 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=20924231&dopt=Abstract https://doi.org/10.1097/EDE.0b013e3181f56757 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=28274907&dopt=Abstract https://doi.org/10.2196/jmir.6978 Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI 5. mHealth Technology Use and Implications in Historically Underserved and Minority Populations in the United States: Systematic Literature Review [Internet]. [cited 2018 Aug 16]. Available from: https://www-ncbi-nlm-nih- gov.offcampus.lib.washington.edu/pmc/articles/PMC6028762/#ref10 6. Fischer BA, Zigmond MJ. 2010. The essential nature of sharing in science. Sci Eng Ethics. 16(4), 783-99. PubMed https://doi.org/10.1007/s11948-010-9239-x 7. Bandyopadhyay S. 2016. S. S. ICT in Education: Open Source Software and its Impact on Teachers and Students. Int J Comput Appl. 151(6), 19-24. 8. Wenger EC, Snyder WM. Communities of Practice: The Organizational Frontier [Internet]. Harvard Business Review. 2000 [cited 2018 Aug 16]. Available from: https://hbr.org/2000/01/communities-of-practice-the-organizational-frontier 9. Lang T. 2011. Advancing Global Health Research Through Digital Technology and Sharing Data. Science. 331(6018), 714-17. PubMed https://doi.org/10.1126/science.1199349 10. Wang J, Abdullah AS, Ma Z, Fu H, Huang K, et al. 2017. Building capacity for information and communication technology use in global health research and training in China: a qualitative study among Chinese health sciences faculty members. Health Res Policy Syst. 15(1), 59. PubMed https://doi.org/10.1186/s12961-017-0222-8 11. Ward JP, Gordon J, Field MJ, Lehmann HP. 2001. Communication and information technology in medical education. Lancet. 357(9258), 792-96. PubMed https://doi.org/10.1016/S0140-6736(00)04173-8 12. McLellan F. 2001. Information technology can benefit developing countries. Lancet. 358(9278), 308. PubMed https://doi.org/10.1016/S0140-6736(01)05534-9 13. Conalogue D, Kinn S, Mulligan J-A, McNeil M. 2017. International consultation on long- term global health research priorities, research capacity and research uptake in developing countries. Health Res Policy Syst. 15(1), 24. PubMed https://doi.org/10.1186/s12961-017- 0181-0 14. Hoffman DM, Blasi B, Ćulum B, Dragšić Ž, Ewen A, et al. 2014. The methodological illumination of a blind spot: information and communication technology and international research team dynamics in a higher education research program. High Educ. 67(4), 473-95. https://doi.org/10.1007/s10734-013-9692-y 15. Moreau M, Asana L, Ngwa W. Information and Communication Technologies Elide Spatial and Temporal Distances to Usher in a New Era of Global Health Collaborations. Proc Am Soc Radiat Oncol 57th Annu Meet. 2015 Nov 1;93(3, Supplement):E382–3. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=21108019&dopt=Abstract https://doi.org/10.1007/s11948-010-9239-x https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=21311011&dopt=Abstract https://doi.org/10.1126/science.1199349 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=28659155&dopt=Abstract https://doi.org/10.1186/s12961-017-0222-8 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11253986&dopt=Abstract https://doi.org/10.1016/S0140-6736(00)04173-8 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11498233&dopt=Abstract https://doi.org/10.1016/S0140-6736(01)05534-9 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=28327164&dopt=Abstract https://doi.org/10.1186/s12961-017-0181-0 https://doi.org/10.1186/s12961-017-0181-0 https://doi.org/10.1007/s10734-013-9692-y Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI 16. Kimball AM, Curioso WH, Arima Y, Fuller S, Garcia PJ, et al. 2009. Developing capacity in health informatics in a resource poor setting: lessons from Peru. Hum Resour Health. 7(1), 80. PubMed https://doi.org/10.1186/1478-4491-7-80 17. Curioso WH, Fuller S, Garcia PJ, Holmes KK, Kimball AM. 2010. Ten years of international collaboration in biomedical informatics and beyond: the AMAUTA program in Peru. J Am Med Inform Assoc. 17(4), 477-80. PubMed https://doi.org/10.1136/jamia.2009.002196 18. Ojwang’ JK, Lee VC, Waruru A, Ssempijja V, Ng’ang’a JG, Wakhutu BE, et al. Using information and communications technology in a national population-based survey: the Kenya AIDS Indicator Survey 2012. J Acquir Immune Defic Syndr 1999. 2014 May 1;66 Suppl 1:S123-129. 19. Waruru A, Achia TNO, Tobias JL, Ngʼangʼa J, Mwangi M, Wamicwe J, et al. Finding Hidden HIV Clusters to Support Geographic-Oriented HIV Interventions in Kenya. J Acquir Immune Defic Syndr 1999. 2018 Jun 1;78(2):144–54. 20. Juma C, Sundsmo A, Maket B, Powell R, Aluoch G. 2015. Using information communication technologies to increase the institutional capacity of local health organisations in Africa: a case study of the Kenya Civil Society Portal for Health. Pan Afr Med J. 21, 23. PubMed https://doi.org/10.11604/pamj.2015.21.23.5130 21. Wekesa M. 2015. Challenges in regulation of biomedical research: The case of Kenya. Niger J Clin Pract. 18(Suppl), S25-30. PubMed https://doi.org/10.4103/1119-3077.170819 22. Ruiz JG, Mintzer MJ, Leipzig RM. 2006. The impact of E-learning in medical education. Acad Med J Assoc Am Med Coll. 81(3), 207-12. PubMed https://doi.org/10.1097/00001888- 200603000-00002 23. Sinclair PM, Kable A, Levett-Jones T, Booth D. 2016. The effectiveness of Internet-based e- learning on clinician behaviour and patient outcomes: A systematic review. Int J Nurs Stud. 57, 70-81. PubMed https://doi.org/10.1016/j.ijnurstu.2016.01.011 24. Munro V, Morello A, Oster C, Redmond C, Vnuk A, et al. 2018. E-learning for self- management support: introducing blended learning for graduate students - a cohort study. BMC Med Educ. 18(1), 219. PubMed https://doi.org/10.1186/s12909-018-1328-6 25. Abraham J. International Conference On Harmonisation Of Technical Requirements For Registration Of Pharmaceuticals For Human Use. In: Brouder A, Tietje C, editors. Handbook of Transnational Economic Governance Regimes [Internet]. Brill; 2009 [cited 2018 Dec 3]. p. 1041–54. Available from: http://booksandjournals.brillonline.com/content/books/10.1163/ej.9789004163300.i- 1081.897 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=19860918&dopt=Abstract https://doi.org/10.1186/1478-4491-7-80 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=20595317&dopt=Abstract https://doi.org/10.1136/jamia.2009.002196 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=26401217&dopt=Abstract https://doi.org/10.11604/pamj.2015.21.23.5130 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=26620619&dopt=Abstract https://doi.org/10.4103/1119-3077.170819 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=16501260&dopt=Abstract https://doi.org/10.1097/00001888-200603000-00002 https://doi.org/10.1097/00001888-200603000-00002 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=27045566&dopt=Abstract https://doi.org/10.1016/j.ijnurstu.2016.01.011 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=30249238&dopt=Abstract https://doi.org/10.1186/s12909-018-1328-6 Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI 26. Lucas RI, Promentilla MA, Ubando A, Tan RG, Aviso K, et al. 2017. An AHP-based evaluation method for teacher training workshop on information and communication technology. Eval Program Plann. 63, 93-100. PubMed https://doi.org/10.1016/j.evalprogplan.2017.04.002 27. Bediang G, Stoll B, Geissbuhler A, Klohn AM, Stuckelberger A, et al. 2013. Computer literacy and E-learning perception in Cameroon: the case of Yaounde Faculty of Medicine and Biomedical Sciences. BMC Med Educ. 13, 57. PubMed https://doi.org/10.1186/1472- 6920-13-57 28. O’Doherty D, Dromey M, Lougheed J, Hannigan A, Last J, et al. Barriers and solutions to online learning in medical education – an integrative review. BMC Med Educ [Internet]. 2018 [cited 2019 Jul 8];18. Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5992716/ 29. Donlan P. 2019. Use of the Online Discussion Board in Health Professions Education: Contributions, Challenges, and Considerations. J Contin Educ Health Prof. 39(2), 124-29. PubMed https://doi.org/10.1097/CEH.0000000000000252 30. Feldacker C, Jacob S, Chung MH, Nartker A, Kim HN. Experiences and perceptions of online continuing professional development among clinicians in sub-Saharan Africa. Hum Resour Health. 2017 29;15(1):89. 31. Osborne DM, Byrne JH, Massey DL, Johnston ANB. 2018. Use of online asynchronous discussion boards to engage students, enhance critical thinking, and foster staff- student/student-student collaboration: A mixed method study. Nurse Educ Today. 70, 40-46. PubMed https://doi.org/10.1016/j.nedt.2018.08.014 32. Steinert Y, McLeod PJ, Conochie L, Nasmith L. 2002. An online discussion for medical faculty: an experiment that failed. Acad Med J Assoc Am Med Coll. 77(9), 939-40. PubMed https://doi.org/10.1097/00001888-200209000-00046 33. Sambunjak D, Straus SE, Marušić A. 2006. Mentoring in Academic Medicine: A Systematic Review. JAMA. 296(9), 1103-15. PubMed https://doi.org/10.1001/jama.296.9.1103 34. Barker ER. 2006. Mentoring--a complex relationship. J Am Acad Nurse Pract. 18(2), 56-61. PubMed https://doi.org/10.1111/j.1745-7599.2006.00102.x 35. Franzblau LE, Kotsis SV, Chung KC. 2013. Mentorship: Concepts and Application to Plastic Surgery Training Programs. Plast Reconstr Surg. 131(5), 837e-43e. PubMed https://doi.org/10.1097/PRS.0b013e318287a0c9 36. Uneke CJ, Ezeoha AE, Uro-Chukwu H, Ezeonu CT, Ogbu O, Onwe F, et al. Improving Nigerian health policymakers’ capacity to access and utilize policy relevant evidence: outcome of information and communication technology training workshop. Pan Afr Med J https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=28445801&dopt=Abstract https://doi.org/10.1016/j.evalprogplan.2017.04.002 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23601853&dopt=Abstract https://doi.org/10.1186/1472-6920-13-57 https://doi.org/10.1186/1472-6920-13-57 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=30998568&dopt=Abstract https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=30998568&dopt=Abstract https://doi.org/10.1097/CEH.0000000000000252 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=30145533&dopt=Abstract https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=30145533&dopt=Abstract https://doi.org/10.1016/j.nedt.2018.08.014 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=12228112&dopt=Abstract https://doi.org/10.1097/00001888-200209000-00046 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=16954490&dopt=Abstract https://doi.org/10.1001/jama.296.9.1103 https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=16460411&dopt=Abstract https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=16460411&dopt=Abstract https://doi.org/10.1111/j.1745-7599.2006.00102.x https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23629123&dopt=Abstract https://doi.org/10.1097/PRS.0b013e318287a0c9 Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI [Internet]. 2015 Jul 23 [cited 2019 Jul 4];21. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4587084/ Appendix A. Learning Objectives, Target Skills Acquired, and Topics Covered Course Title Learning Objectives Target Skills Topics & Activities Covered Creative Integration of ICT Tools and Technologies for Enhancing Research Design, Management and Implementation (Train the Trainers Module as well as introductory module for the other courses) • Understand the Role of ICT Tools in Responding to Health Research Challenges • Use ICT tools to finding and managing the Scientific Literature; • Discuss and use Social Media • Use Data Visualization tools • Apply ICT tools in Scientific Communications • Apply Research Productivity tools • Use various Mobile Technologies and Tools • Use ICT tools for Project Management. • Retrieve relevant research articles from appropriate research databases to support the development of research hypotheses and research proposals. • Use a bibliographic management tool (Mendeley or similar) to download and management research citations and full-text articles • Identify and select relevant mobile technology tools to design and implement a simple research data collection instrument. • Use a variety of desktop productivity tools in interactive ways to enhance data analysis and reporting. • Use tools and technologies to improve research • The Role of ICT Tools in Responding to Health Research Challenges • Finding and Managing the Scientific Literature • Social Media • Data Visualization • Scientific Communications • Research Productivity • Mobile Technologies and Tools • Project Management https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4587084/ Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI communications including written and oral presentations. • Demonstrate creative integration of disparate ICT tools to support all phases of the research process. Geographic Information Systems • Recognizing the growing significance of spatial analysis and Geography for advancing health research • Describe how GIS can be used to bring together mapping, data management, and geospatial analysis techniques in the context of health research • Understand how to use GIS as a tool to perform basic quantitative geographic analysis and spatial data visualization (mapping) • Recognize the unique challenges of performing quantitative analysis with spatial data • Assess the use of GIS in the research literature of any knowledge domain/discipline • Understand the components of a geographic information system (GIS) and how it can be used effectively throughout scientific research • Competency using Quantum GIS (QGIS), an open source GIS software package, to merge, manipulate, analyze and map spatial data • Ability to locate spatial data of the correct type and quality, in order to address a research question • Comprehension of basic Cartographic Design (Mapping) skills in order to clearly communicate results to your audience • Understand how GIS brings together mapping, data management, and • What is Geography and GIS?: Exploring and interpreting web mapping systems • Learning about GIS software: Installing and Navigating QGIS • GIS data types: Importing data layers and basic mapping • GIS data sources: Key resources for spatial data and quality considerations • Spatial analysis: Working with analysis functions in QGIS • Cartography: Design and formal map layout in QGIS • GIS Research workflows: Evaluate the use of basic GIS techniques in other’s research Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI • Develop and implement spatial research questions using GIS geospatial analysis technologies for health research Principles & Practice of Research Data Management • Understand the unique ethical and technical requirements of research involving human participants • Know the historical evolution of methods for managing clinical research data • Know types of data management software that are appropriate for single site and multicenter research, for studies of varying size and complexity • Understand the characteristics of successful research data management operations • Know how research data audits are conducted and how to prepare for one • Understand methods for ensuring the security and confidentiality of physical and electronic research data • Understand the strengths and • Managing research data in compliance with international Good Clinical Practice standards • Data modeling for design of computerized databases that accommodate sparse data and repeated measures • Ability to create processes that result in research data that is accurate, complete, timely, verifiable, secure, and available for analysis • Ability to write a complete research data management plan for a clinical research project, including process, people, technology, and budget components • Ability to design high quality paper and electronic data capture forms • Ability to use two contemporary online data management systems (REDCap • What is Biomedical Informatics, and how does it contribute to effective data management? • Principles of observational and interventional research involving human volunteers • Characteristics of ‘sensitive’ data, it’s acquisition, storage • What are International Good Clinical Practice Standards and why are they important? • Planning sequence for designing and implementing a research study and its data management infrastructure • Strengths and weaknesses of hierarchical, object- oriented, XML and relational database technologies • Data coding standards to maximize utility and re-usability of data • Designing data capture forms to Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI hazards of using the Internet for acquisition and management of sensitive data and Open Data Kit - ODK) to design and implement a simple research study maximize usability and data completeness • Specialized contemporary electronic data capture technologies, including barcoding, Scantron and Teleforms. • Design and use of smartphone apps for research data management Research Management & Communication Tools • Describe the skills used in the research process and link with a highlight of the research process steps already covered in previous courses such as data management. • Find and use a data set to do basic data analysis focusing on descriptive statistics, and basic parametric tests. • Perform data analysis using descriptive statistics and basic parametric tests using an open source data analysis software such as PSPP. • Describe results dissemination • Understand the research process. • Demonstrate how to identify data sets. • Demonstrate knowledge of descriptive statistics, and basic parametric tests • Understand how to use a statistical tool to analyse data to generate results for descriptive statistics, and basic parametric tests • Understand how to make oral presentations and write research papers • Understand how indexing and web searching and • ICT skills used in the research process and link with a highlight of the research process steps already covered in previous courses i.e. introduction to the research process, data management and data visualization • Data analysis. Identify a data set and guide the trainees on how to do basic data analysis, perhaps focusing on descriptive statistics, and basic parametric tests. This will be done as theory. • Data analysis using descriptive statistics Improving Information and Communications Technology (ICT) Knowledge and Skills to Develop Health Research Capacity in Kenya Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 11(3):e22, 2019 OJPHI techniques such as presentations, policy briefs. • Describe how research collaboration can occur using LinkedIn as a tool, and one other tool for collaboration. • Demonstrate how indexing and web searching and bibliography is done. bibliography are done. and basic parametric tests using an open source data analysis software such as PSPP. The learners should run some designed tests to see some sample results screens displayed, and then they run tests as assignments • Results dissemination techniques: presentations, policy briefs, etc • Research collaboration, using LinkedIn as a tool, and one other tool for collaboration • Handling of indexing and web searching and bibliography