key: cord-328315-idel6l11
authors: Mellor, Nicholas; Horton, Hetty; Luke, David; Meadows, Jon; Chatterjee, Arunangsu; Gale, Thomas
title: Experience of Using Simulation Technology and Analytics During the Ebola Crisis to Empower Frontline Health Workers and Improve the Integrity of Public Health Systems
date: 2016-12-31
journal: Procedia Engineering
DOI: 10.1016/j.proeng.2016.08.062
sha: 
doc_id: 328315
cord_uid: idel6l11

Abstract The Ebola outbreak highlighted the challenge of health security and particularly of how best to give frontline workers the knowledge, confidence and competence to respond effectively. The goal was to develop a tool to improve infection prevention and control through local capacity building within the context of an emergency response. The research showed that digital technology could be a powerful ‘force multiplier’ allowing much greater access to high fidelity training during an outbreak and keeping it current as protocols evolved or new safety critical steps were identified. Tailoring training to the local context was crucial to its relevance and accessibility. This initiative used a novel approach to the development of the training tool – ebuddi. It used agile development to co-create the tool with active participation of local communities. A further pilot showed how it could be extended to meet the longer term needs of triage training and ensure better quality assurance. In the longer term it may have the potential to improve compliance with International Health Regulations, be adapted for future emergencies, and contribute to global health security.

The recent Ebola outbreak highlighted four key areas of concern in health security. Firstly, it focused attention on the importance of infection prevention and control (IPC) competence and compliance amongst frontline workers in a region as large and remote as West Africa. Secondly, training needed to be accessible to a wide range of people be they working in healthcare and the laboratory sectors or burial teams and maintenance staff -all with large variations in skills, knowledge and education levels [1] . These workers, whether they were professionals or volunteers, needed access to training to keep themselves, their patients, colleagues and families safe. Thirdly, community engagement was vital in controlling the outbreak. Finally, the need to be prepared for novel and new pathogens such as Middle East Respiratory Syndrome Coronavirus (MERS-CoV), Severe Acute Respiratory Syndrome Coronavirus (SARS-CoV) and other strains showing resistance to anti-microbials. Ebola was not a one off -there will be constant challenges of this nature, requiring healthcare systems to be more resilient, watchful and responsive than ever before. There are important messages to learn about health security from the Ebola response, both in the West African region and globally. Even highly trained international health workers lacked skills in the use of Personal Protective Equipment (PPE) which lead to high profile cases of infection. These observations imply a problem with how such epidemics are handled and without a change in attitude and approach; similar issues could resurface in the next outbreak, as illustrated by the challenge of dealing with the endemic Lassa Fever in the same region [2] .

In his article, 'The Next Epidemic -Lessons from Ebola', Bill Gates emphasised the need for better and faster training for health personnel to confront and contain an epidemic quickly [3] . He is not the only figure to draw attention to training on a global stage. Multiple reviews reflect lessons learnt from community empowerment and giving the frontline the tools, knowledge and competences to contain the outbreak [4, 5] . This a lesson for all countries as highlighted by the EFN report on EU Health Professionals' Perceptions of Preparedness for Ebola and Infectious Diseases of High Consequence (IDHC) 'We are not prepared unless we are all prepared' [6] . In addition there was the recognition of the importance of innovation and finding smarter ways of responding, but most initiatives struggled to realise their full potential [7, 8, 9] .

Simulated training can 'bring learning alive' and increase engagement levels [10] . It encourages teamwork, in particular the role of the buddy, which is vital in situations where PPE is required. The digital nature facilitates a network-based approach to distribution allowing tight version control and replication across communities, districts, countries and potentially continents [11, 12] . It helps shift training from a classroom based, didactic style delivered by an international expert to a more peer based, personalised approach that overcomes some of the constraints inherent with a centralised, cascade based approach to rolling out new training.

This article presents a model that could transform IPC training. It blends traditional training with tablets to create a technologyenhanced approach that aims to improve patient and health worker safety. The system, known as ebuddi, has been prototyped in West Africa and has the potential to improve quality, accessibility, scalability and legacy of training [10, 13] . This novel approach has attracted international attention and it could be used to improve health systems resilience and outbreak response worldwide.

Creating the ebuddi prototype was a collaborative approach between organisations with frontline operational capacity and specialists in medical education, training and agile software development. The Masanga Mentor Ebola Initiative (MMEI) comprised the Masanga Hospital in Sierra Leone, The MENTOR Initiative and experts from Merlin's Lassa Fever Programme. Plymouth University Peninsula Schools of Medicine and Dentistry (PUPSMD) were involved throughout providing input based on pedagogical principles and best practice in virtual learning and distributed simulation.

The core concepts were based on national guidelines and IPC curriculum developed by the Liberian Ministry of Health in association with the WHO and CDC. These were applied to pedagogical mechanics from game based learning and used agile development to interpret them with Graphical User Interface (GUI), animation and language [13] . Initial prototypes were built from videos with WHO expert trainers and photos of the healthcare settings, and evolved through co-creation and agile development. Co-creation by experts and frontline trainers produced an authentic module that engaged the audience more than passive illustrations portrayed in textbooks or demonstrations by international experts alone. Agile development meant that frontline trainers and healthcare workers could feedback to the international development team and see their involvement included in subsequent builds.

Initially the development involved input from diaspora groups able to meet directly with the development team in London. However as communications links were put in place with operational training teams on the ground, input to the development team could be provided directly from those in the frontline -ensuring even greater authenticity and relevance of the training material to the safety critical steps. Fig.1 illustrates this continuous loop of communications between the in-field metrics and international development teams exchanging module iterations with data to improve technical performance and learning impact. The quicker this feedback was incorporated, the more impact it had on subsequent suggestions as trainers were encouraged to see the value of their ideas, encouraging bottom up innovation.

Open innovation and collaboration make digital approaches to learning more accessible and affordable. For such technologybased initiatives, insights such as the Principles for Digital Development act as 'living guidelines that can help development practitioners integrate established best practices into technology-enabled programs' [13] . Considering an open approach to technology-enabled international development would encourage a free flow of ideas that permeate organizational boundaries, not waste public resources unlocking code and duplicating work. The Principles guide strategies for leveraging and contributing to broader resources and knowledge to give greater impact. With these guidelines, we can make a more concerted effort to institutionalise the many hard lessons learned in the use of information and communication technologies in development projects [14] . 

The field trials comprised quantitative and qualitative studies to investigate the value of ebuddi. They introduced the concept of IPC and PPE, as well as the effectiveness of trainers, to healthcare workers both during the acute phase of the emergency and the subsequent transition into restoring essential healthcare. The training tool was developed by Immerse Learning, PUPSMD and Total Monkery, and a team of experts led by Jon Meadows. Fieldwork was conducted by Masanga Hospital and The MENTOR Initiative; supported by expertise at PUPSMD. The training team comprised IPC specialists within The MENTOR Initiative who, during the course of the studies, visited over one hundred health facilities. The majority of these facilities were private health clinics and health centres, in addition to three hospitals and one Ebola Treatment Unit (ETU). Facilities covered both urban and rural areas. Close to five hundred healthcare workers were introduced to the tool during the study.

The fieldwork comprised four main phases: Phase 1: Introducing the concept. Early prototypes developed by Immerse Learning were introduced to healthcare workers in Sierra Leone. Feedback confirmed that the tool was relevant and appropriate for training healthcare workers and supported the model which incorporated training in both English and local language.

Phase 2: Testing potential. Conducted in health facilities in Liberia to test the potential of the program. Small groups used ebuddi and gave informal feedback and first impressions. It was tested as a standalone tool where participants used the module on a laptop for approximately one hour, independent of scheduled training, and evaluated confidence levels before and after.

Phase 3: Testing efficacy. The study used rapid and agile feedback mechanisms to accelerate development and prioritize local input. Each participant evaluated their confidence with IPC practices and then was assessed on their competence at donning and doffing enhanced PPE. ebuddi was used to reinforce the practical experience and then the participant was reassessed on their practical skills. Quantitative data collection identified trends and performances.

Phase 4: Operational value. This phase focused on how best to apply ebuddi in different contexts -targeting end users with a range of resources and abilities. Field trials integrated ebuddi into scheduled group IPC training, focusing on PPE training. The program was used alongside practical demonstrations and skill stations, continuously adapting to meet changing resources.

The studies gave a real insight into the concept of ebuddi -how it might be best used in the field and its potential as a teaching tool to support traditional training methods. There were many positive attitudes across the different stakeholders shown in Fig.2 , although the multiplicity of stakeholders meant that building a coherent base of support was often a slow process given their different perspectives and priorities when it came to training. 

The initial feedback was encouraging and people enjoyed the concept. However, most users had minimal experience with laptop technology and initially required constant guidance from the trainer on how to operate the module, especially on the first run. The unfamiliarity of the target audience with computers meant that many users struggled to use a mouse. This contrasted with their confident use of smartphone technology and touch screen interfaces. West Africa has seen technology leapfrog from minimal use of social media and connectivity to rapid adoption of to smartphones and use of tablets, without following the evolutionary path through desktops and laptops. This meant that agencies outside the region often underestimated the utility of digital tools or were constrained by the belief that custom low cost hardware would be necessary to ensure the broadest possible update of such tools.

Introducing tablets enhanced trainee engagement as well as GUI intuitiveness. With the ever-increasing prevalence of smart phones, a touch screen interaface was found to be more and more accessible. This shifted the study focus from laptops to Android based tablets due to their size, relatively low power consumption and usability. The intuitiveness of the module and the familiarity of the graphics were important. Unknown objects in the background distracted users from the key messages from the program, so the study worked alongside Liberian trainers to design a setting that was familiar to end users. Including local voiceover (language and dialect) increased understanding. Software developers focused on making the module as intuitive and graphical as possible, recognising that some target users are illiterate, let alone computer literate.

Initial wariness to foreign schemes pervaded the communities throughout the emergency response but this authenticity, as well as trust in the local training teams, helped overcome this challenge. The close involvement of Sierra Leonean and Liberian experts ensured a more authentic tone and familiar setting to the content being developed.

Testing ran concurrent to software development. Daily updates facilitated rapid feedback direct to developers, made possible through digital communications and the adaptability of the simulation environment compared with video based training materials.

It was important to value and input suggestions from frontline trainers, and agile development allowed this to happen quickly. Once trainers could see their suggestions incorporated into new versions, it encouraged further feedback and a sense of co-creation -a virtuous circle of engagement. However, this process was not without its challenges -from the most basic access to sufficient connectivity for conference calling, through to information management and prioritisation of development recommendations. A key lesson learned was the need to communicate clearly to the front line healthcare workers about what development would be enhanced for the next app release. Their expectation of instant and specific modifications from their personal recommendations grew quickly and at times to an unrealistic level.

The training had to be flexible and adapt to changing resources, including time constraints, staff availability, limited equipment and power management. The advantage of ebuddi was that as modules were developed they could be added as additional posters in the simulated clinic, as shown in Fig. 4 . In the end, this would allow trainers to pick and choose the topics they needed according to their local priorities. The initial ebuddi module evolved to respond to the changing training needs in Liberia. Its teaching responded to changes in the National Guidelines that shifted focus from specific Ebola material to generic IPC in health facilities -from the 'Keep Safe Keep Serving' curriculum to the 'SQS' curriculum [15, 16] . The Liberia specific module developed is currently being adapted for training in standard precautions and IPC in Sierra Leone to build resilience post Ebola as part of the iCARE project, funded by the Department for International Development UK. Agile development has enabled the core modules to be adapted to this changing focus. This was a good indication of how the program could adapt to new material, something that the modular approach helped significantly.

The studies demonstrated the value of near peer learning when introducing new concepts. Collaboration and teamwork were important and emphasised in the conversations during the introduction of the blended approach to learning. This immediate feedback identified areas where training needed to be reinforced to avoid common or dangerous mistakes.

Some unexpected findings came out of the fieldwork combining digital technology with peer learning in austere environments. In one example a nurse, who was confident with IPC was struggling to use a tablet. She was helped by a nurse aid who was more computer literate yet did not know much IPC. This collaborative approach to learning between the trainees was insightful and very encouraging to watch. 

Inbuilt data collection was trialled in the latter stages of the fieldwork. The module introduced a personalised user record that tracked which topics had been covered and the scores achieved in each competency. Users liked to know their scores and progress through the module and so the data collected could be used for personal feedback as well as wider progress tracking. It could facilitate targeted coaching to individuals and also more novel outreach functionality, for example, if a key module was overlooked, it would be flagged and the coordinator could remind the trainer responsible for that facility.

The field trials in West Africa tested the ebuddi prototype and explored its potential in different and challenging environments. It began to address some of key concerns in health security raised from the lessons learnt from the Ebola response -focusing on improving outcomes in skill improvement not just outputs of number of health workers trained. The importance of IPC in such outbreaks is undeniable. Frontline workers need frequent, high quality training that is accessible to all and independent of location and education level. It needs to adapt to changing resources such as available equipment and how to respond to shortages, introduction of new equipment, procedural change and learning from incidents that might have resulted when critical safety steps have not been fully understood. The training needs to adhere to local policies and protocols. This is captured in the ebuddi development process by a compliance document that details local guidance and captures key learning, critical steps and evaluation points. The field trials showcased the potential of ebuddi to augment existing IPC training in Liberia, which has contributed to the success of stopping the outbreak. True behavior change of health care workers is difficult to ensure as adherence to standard protocols can wane quickly when the acute threat of infection is deemed to have reduced. The second key concern is that everyone must have access to required knowledge and skills to work with confidence and competence, a philosophy enshrined in the EFN report 'We are not prepared unless we are all prepared' [6] . ebuddi seemed to increase the confidence of healthcare workers in the short term, although the long term benefits require further research. A key finding from the fieldwork was that the graphics needed to be familiar to the trainee -including the design of the health facility, equipment available and surrounding landscape (rural or urban). The addition of a local soundscape further increases the immersive nature of the learning experience. Increased usability of the module developed from feedback, engaged a wide audience regardless of skills, knowledge or education levels. Community engagement is vital in controlling an outbreak such as Ebola. The co-creation model of ebuddi empowers frontline workers and trainers to give valued feedback and critically be able to see their input in the module. This ensured authenticity and buy in to any subsequent development. The avatar characteristics can be changed to reflect an appropriate mix of gender, ethnicity and culturally appropriate clothing. This gives the module great potential to relate to different communities and demographics, and recognise the importance of highlighting the key role women play in the response by highlighting this in the simulation exercises [17] .

Preparedness for fresh outbreaks from existing pathogens such as Ebola or Lassa fever or emerging pathogens such as Middle East Respiratory Syndrome (MERS) is important. This is where the agile development of ebuddi can really rise to the challenge as it has the potential to respond quickly and accurately to an outbreak with international experts co-creating training with local community actors. Using analytics and data to collect information and statistics can be used to identify trends, gaps through automated monitoring, and evaluation of an outbreak much faster and at higher integrity than before. ebuddi could ensure frontline health workers have the right kind of training in time, acting as an aid to programme management of resources, as well as provide direct support to health trainers carrying out each session.

There was scepticism whether such research could be carried out during an emergency. Some frontline agencies saw a technology-based approach as an additional complication, cost factor or constraint in training programmes. However, the MMEI partnership illustrated the capacity of small organisations or alliances to be more responsive and innovate more easily than larger, more bureaucratic organisations -although they often face challenges in getting support from donor agencies who often prefer to support well established implementing partners with whom they have an existing relationship [9] .

In the field of health research the main focus has been on therapies, vaccines and equipment where there are good precedents of public private sector partnerships leading to tangible products. There are fewer precedents in the digital field possibly leading to less appreciation in the emergency sector of how significant such technology could be in re-imagining capacity building. The business sector has already seen how digital simulation technology can be delivered on multiple platforms where laptops, tablets and mobile phones could all provide an opportunity to teach and train; and the growing trend for people to bring their own device (BYOD) to the training session.

Every crisis is an opportunity to innovate, build on lessons learnt and share new sights and better practices more widely. The importance of doing this was recognised in the European Summit meeting on lessons learnt from the Ebola response which concluded that successful innovations from the recent Ebola response need to be built on to create 'a smarter, more scalable and sustainable' response capability in the future [18] .

The Wilton Park meeting (2015) on empowering frontline health workers concluded: 'this is a moment in time. The last 15 years have seen a revolution in ICT and mobile technologies. Ebola shone a spotlight on the ineffectiveness of past health systems strengthening efforts; there is growing evidence that ICT and mobile are a vital part of the solution to build resilient health systems.' [19] .

The field trails demonstrated that it was possible to innovate in an emergency response setting. It had many challenges and it was important to reduce the burden of the technology where possible. Nonetheless, the benefits of the analytics and the inbuilt monitoring and evaluation could, in the long run, make the data collection process much less burdensome.

ebuddi was designed to provide a legacy to the Ebola outbreak -an expert training aid with the potential to be adapted as practice, protocols and equipment evolved. Right from the start it aimed to improve the emergency response and to leave a legacy that would help to ensure a higher standard of training after the emergency was over.

The transition from a conventional training model to a blended or technology enhanced model is often difficult, but if this path can be successfully navigated then MMEI's research demonstrated many benefits of using digital technology for training in an outbreak response. These include using a standardised training format to promote key safety protocols and the ability to detect lack of knowledge and skills through analytics embedded in the tool. The use of tablet devices proved transformational compared with early trials that used laptop computers -in particular the usability of the interface, their intuitiveness and lower power consumption.

Bringing together the expertise and resources for an effective blended learning programme required a breadth of skills and investment beyond any single organisation. The MMEI partnership not only was able to draw together such competences but also benefit from the insight and expertise of a wide range of volunteers from a wide range of specialities, sectors and countries around the world keen to contribute to the fight against Ebola. ebuddi could assist a coherent approach to evaluating outbreak response competences such as the standards for outbreak response set by the International Health Regulations (IHR) [20] . The IHR represents the agreement between 196 countries including all WHO Member States to work together for global health security and stipulates that each country needs a human resource development plan to address the gaps existing between the knowledge and skills required to comply with IHR requirements and the knowledge and skills available in the workforce. Currently compliance with IHR is often difficult to assess.

Longer term, the ebuddi model could be adapted to other regions and public health contexts where there is a need to boost local understanding of infection, prevention and control as well as personal and patient safety. The adaption process would require alterations across three areas -protocol adherence, graphical interface and avatar characteristics. The architecture of the code underlying the module has been built in a way which allows continual development.

The advanced analytics could enable real-time tracking and monitoring of the training impact. It could be used to improve training in real time, offering quality assurance to trainers and project managers on the ground through automatic monitoring and evaluation, thereby improving the integrity of the public health training programme. The analytics may be presented on an individual basis, of a training group, within a community, district, national or international level. On a wider stage, data provides a greater level of transparency for donors and other sponsoring agencies. Analytics will ensure continuous improvement, building the evidence base to support future deployment as well as being a catalyst to continuing innovation.

As the tool matures, ebuddi may be able to provide high quality training in hard to reach places, recognising the physical constraints of movement during a public health crisis, geographical inaccessibility, or the threat of insecurity. The study has shown ebuddi could enhance conventional approaches to local capacity building by improving training effectiveness, increasing cost efficiency when scaled and enabling an agile response to changing priorities.

Ebola -what went wrong? London. The Lancet Global Health Blog

Lassa Fever Update

The Next Epidemic -Lessons from Ebola. USA

A community-engaged infection prevention and control approach to Ebola. Health Promotion International

Community-Centered Responses to Ebola in Urban Liberia: The View from Below

We are not prepared unless we are all prepared' EU Health Professionals' Perceptions of Preparedness for Ebola and Infectious Diseases of High Consequence

One year into the Ebola epidemic. World Health Organisation

Case study: Innovations in Emergency Disease Responses. Centre for Research in Innovation Management (CENTRIM)

What we've learned about fighting Ebola

Virtual Learning And Distributed Simulation (V-LADS) for preparing healthcare workers at peripheral health units to protect themselves against Ebola Virus Disease (EVD) in West Africa

Advancing today's training and tomorrow's outbreak preparedness: the importance of innovation

A networked approach to improving the resilience of communities confronted by the threat of Ebola. London. The Lancet Global Health Blog

Health worker focused distributed simulation for improving capability of health systems in Liberia. Simulation in Healthcare

Digital Development Principles Working Group. The Principles for Digital Development

Ebola Virus Disease (EVD) Infection Prevention and Control Standard Operating Procedures (SOP) For Health Clinics, Health Centers and Hospitals

Liberia Ministry of Health IPC Task Force

The Psychosocial Aspects of a Deadly Epidemic -Women in the Ebola Crisis: Response and Recommendations from UN Women

Conference: Lessons learned for public health from the Ebola outbreak in West Africa -how to improve preparedness and response in the EU for future outbreaks

Re)Building health systems in West Africa: what role for ICT and mobile technologies? (WP1409)

Strengthening health security by implementing the International Health Regulations. Geneva. World Health Organisation

The authors are grateful to the following partners, contributors and organisations for their continued help and support through this programme: The Telegraph Christmas Appeal, Richard Allan, Geoff Eaton, Elton Gbollie, Dr Simon Mardel, Dr Jurre van Kesteren, Dr Bart Waalewijn, Dr Austin Hunt, Richard Scott and the IPC team in the The MENTOR Initiative -Liberia programme.
