ISDS Annual Conference Proceedings 2018. This is an Open Access article distributed under the terms of the Creative Commons Attribution-
Noncommercial 3.0 Unported License (http://creativecommons.org/licenses/by-nc/3.0/), permitting all non-commercial use, distribution,  
and reproduction in any medium, provided the original work is properly cited.

ISDS 2018 Conference Abstracts

Approaching Evaluations of Surveillance System 
Pilots through an Ownership Perspective
Carrie Eggers*
Center for Global Health, CDC, Atlanta, GA, USA

Objective
We used experiences in multiple countries to determine that owner 

engagement is critical for successful evaluations of surveillance 
system viability.

Introduction
Pilot projects help determine utility and feasibility of a system, 

but even if considered successful, cost could prevent further scale-up. 
When evaluating a surveillance system pilot, cost and benefits are 
key factors to examine. In Cote d’Ivoire and Tanzania, Ministry of 
Health (MoH) and non-governmental partners receive funding under 
the Global Health Security Agenda to strengthen disease surveillance 
for earlier detection and improved response to potential infectious 
disease outbreaks. To this end, Community based surveillance (CBS) 
projects were implemented in 2016 as a means for early warning of 
potential events to facilitate a more rapid response.

Currently, these CBS projects are being evaluated collaboratively 
with the primary stakeholder, the host country government (HCG), 
as lead, and partners such as CDC providing technical assistance. 
In other instances, partners may conduct an evaluation and share 
the results and recommendations with the HCG; however, if the 
HCG is not actively engaged as the primary executor, outcomes 
may not be endorsed or implemented. Therefore, these evaluations 
were approached from an owner’s (HCG) perspective. In this way, 
the governmental agencies develop capabilities to conduct similar 
activities in other areas, reduce dependencies on outside entities, and 
promote enactment of resulting recommendations.

Methods
Once the determination was made that an evaluation was necessary 

to decide the usefulness of the projects for future planning, key 
stakeholders worked together to design and execute the evaluation. 
For Cote d’Ivoire, the evaluation team consisted of representatives 
from the MoH’s National Institute of Public Health, Directorate 
of Informatics and Health Information, and Directorate for the 
Coordination of the Expanded Immunization Program, along with 
delegates from CDC and implementing partners. In Tanzania, 
evaluation team members came from the MoH, the Ministry of 
Agriculture, Forestries and Livestock, WHO, CDC and implementing 
partners. Team members participated in either planning, conducting 
or analyzing the evaluation, while some contributed to a combination 
or to all aspects. MoH members led the effort with CDC and other 
partners providing technical assistance, while implementing partners 
contributed only to planning and logistics to reduce the potential for 
bias.

For the initial step, representatives came together to fully document 
the system to be evaluated. This system description details the 
purpose, relevant stakeholders and current operation of the pilot 
system. As the evaluation question should remain within the scope of 
the system’s purpose, it was necessary to definitively understand and 
confirm the goal and objectives set out for the system. Next, the sites, 
participants and roles, and data flows were described, noting that 
verification of the actual processes would occur during the site visit 
portion of the evaluation. Total cost of ownership was calculated by 
considering solution costs, implementation costs and ongoing support, 

and then broken down by district. The CBS pilots implemented early 
warning notification systems in two districts in Cote d’Ivoire and in 
five districts in Tanzania using a combination of paper-based and 
electronic reporting formats.

Evaluation teams visited pilot sites and routine surveillance sites for 
comparison and conducted in-person interviews using questionnaires 
specific to the individual’s role. Data were either collected in the field 
on paper forms or electronically on tablets for subsequent upload 
to a centralized database for later analysis. Data from project and 
routine reporting databases were comparatively analyzed to calculate 
timeliness, validity, usefulness, acceptability and value of the early 
warning system pilots.

Results
Although final interpretations of the evaluation results are 

pending, the evaluations were successfully led by the HCG and 
jointly conducted with other stakeholder engagement. Leadership 
by the owners of the systems has already resulted in the recognition 
that certain aspects of the pilot surveillance systems demonstrate 
a successful and affordable approach, while others will need to 
consider more cost-effective strategies. Though further analysis will 
likely continue to show the utility of CBS strategies, the ownership 
approach is resulting in an outcome of broad stakeholder input with 
approval from the host country government.

Conclusions
Community based surveillance can help to detect events of public 

health importance and effect earlier introduction into the health 
system for more timely situational awareness and response. However, 
it is difficult to determine the costs associated with different strategies 
of implementation and operation in order to ascertain the value for 
public health action. Additionally, pilot implementations of these 
systems are often funded at a level that cannot be replicated nationally 
and not for a prolonged period of time. While it is believed that CBS 
can be a cost effective early notification system, continual monitoring 
and routine evaluation is required. By routinely monitoring cost and 
quality, sustainability of the system can be continually assessed 
and system adaptations made accordingly. Key to remember is 
that evaluation must occur from an owner’s perspective and must 
engage the people who are going to govern, operate and provide the 
ongoing resources for system operation. In this way, effectiveness and 
efficiency can be continually monitored within the parameter of cost 
so that viability of the system can be ascertained.

Keywords
Community based surveillance; Evaluation; Stakeholder engagement; 
global health

*Carrie Eggers
E-mail: ceggers@cdc.gov

Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 10(1):e169, 2018