HIV 883

CONFERENCE REPORT

‘Feedback: Where data finally get
thrilling’ – tools for facility managers to use data

for improved health outcomes in the prevention of
mother-to-child transmission of HIV and antiretroviral therapy

J Murphy,1 MPH; C-H

Mershon,2 MPH;
H Struthers,1 MSc, MBA; J McIntyre,1,3 MB ChB, FRCOG

1
Anova Health Institute, Johannesburg, South
Africa

2
Gillings School of Global Public Health,
University of North Carolina at Chapel Hill, USA

School of Public Health and Family
Medicine, University of Cape Town, South Africa

Corresponding

author:
J Murphy
(murphy@anovahealth.co.za)

Data use and data quality continue to be a
challenge for government sector health facilities and districts
across South Africa. Led by the National Department of Health,
key stakeholders, such as the Anova Health Institute and
district health management teams, are aligning efforts to
address these gaps. Coverage and correct implementation of
existing tools – including TIER.net, routine data collection
forms and the South African District Health Information System –
must be ensured. This conference report provides an overview of
such tools and summarises suggestions for quality improvement,
data use and systematic evaluation of data-related
interventions.

S Afr J HIV
Med 2013;14(3):131-134

DOI:10.7196/SAJHIVMED.883

There is increasing recognition of the
importance of a functional information-management system to
improve health outcomes in South Africa (SA). This is gaining
attention through a number of local and international policy
documents, including the SA District Health Management
Information System (DHMIS) Policy (2011),1 the Aid
Effectiveness Framework (2012) 2 and the
US President’s Emergency Plan for AIDS Relief (PEPFAR)
Partnership Framework.3 With ongoing evaluation and
improvement of the SA District Health Information System (DHIS),
patients, clinicians and policymakers are ideally positioned to
benefit from the improved quality and increased use of routinely
collected data at facility, sub-district and district levels. In
the case of HIV services, the DHIS can be particularly valuable
in determining the number of clients receiving antiretroviral
therapy (ART) and in identifying gaps in the prevention of
mother-to-child transmission (PMTCT) of HIV services.

The Anova Health Institute
(Anova) recently gathered 160 delegates in Johannesburg for
the symposium ‘Feedback: Where data finally get thrilling’,
to provide an overview of best practice for information use in assessment and improvement
of health services, with an emphasis on HIV treatment and
PMTCT. The target audience included facility managers across
Gauteng Province, with a focus on Johannesburg. Anova partnered
with information and programme managers from provincial and
district government, as well as a variety of non-governmental
organisations (NGOs), to maximise expertise and objectivity on
the issue.

Magnitude of the issue

The DHMIS Policy calls for more than just
addressing data quality; it denotes that information should be
used in programme planning and in clarifying the main roles and
responsibilities ‘for ensuring data completeness, data quality,
and data use and “ownership” at all levels of the health
system.’4
One finding of this symposium was voiced by those in attendance:
the DHMIS Policy is not available or followed by all facility
managers, especially in the areas of data use for programme
decisions and feedback between all levels (facility to
sub-district/district and vice versa).

The DHIS,
which since 1996 has been the sole government repository of
health-related data in SA, has not reached optimal levels of quality, as
documented5 and
anecdotally reported. This holds true for PMTCT as documented
by Mate et al.,6 as well as for
ART data which are not as well documented. Particular areas of
concern include data accuracy, completeness and reliability.
Fortunately, the National Department of Health (NDoH),
facility managers, district DoH structures and NGO partners
have begun the implementation of tools like TIER.net, the
Prevention of Mother-to-Child Transmission Action
Framework and the District Health Barometer (DHB) to
interrogate and better utilise information. In this context,
this conference report is not a declaration of success, but rather a
brief description of the status of our progress in using tools
to strengthen data quality and ease of use.

Conference content

Keynote speaker, Winnie Moleko from the
Wits Reproductive Health and HIV Institute (WRHI)/NDoH,
presented ‘Data feedback towards quality improvement in
service delivery’. Moleko discussed the state of data quality
in SA and the role that this plays in quality improvement and
implementation of the National Core Standards.7
Practitioners and policy makers can use data to identify gaps
in service delivery, resources and facility needs. For data to
be useful, they must be correct and accurate;
data that are incorrect or presented misleadingly can be
detrimental to service delivery and planning. One suggestion
that Moleko made, which can be implemented in service
facilities, is to post the facility’s data on improvements and
achievements in a public place in the facility. This allows
staff and clinicians to engage the public and clients in the
facility’s data-improvement process.

All presentations are available online
(http://www.anovahealth.co.za/resources/entry/feedback_where_data_finally_gets_thrilling/
). Table 1
summarises the
lessons learnt for clinicians and facility managers working in
the field of HIV. The body of the symposium covered three main
areas: (i) review of data quality and
challenges; (ii) best practice review of data
use for quality improvement; and (iii) data tools available to
facilities, clinicians and policy makers.

Review of
data quality and challenges

Mokgadi Morokolo represented
Johannesburg Health Information and gave an
overview of the DHMIS. She reminded the facility managers in
the audience of their responsibility for the data signoff
process. This includes a review of the source data such as
facility registers, critical analysis of the data outputs, and
timely submission of reports and corrections. She emphasised
that this is the responsibility of sub-district
managers, district directors and hospital chief executive
officers (CEOs). These managers are also responsible for
improving their knowledge of indicators and maintaining
current data-collection tools. District directors are
responsible for ensuring that facilities have the current and
correct stationery.

Goodwill Kachingwe and Nowinile Dube presented recent
district-level data and highlighted where data can and should be
used at all phases of the programme cycle (Fig. 1). Data are
used in the conceptual phase to help determine what health
outcomes need to be addressed through the programme. Data can be
used in the planning phase to provide insight into where
resources need to be distributed or to provide a baseline for
future evaluation. In the implementation phase, data are used to
monitor the programme implementation or to ensure that target
populations are being reached by the programme. In the
termination phase, data are used to evaluate the success of the
programme, or to determine how the programme has contributed to
district, provincial or national targets.

Fig. 1. Flowchart indicating where data
can and should be used at all phases of the DHMIS programme
cycle.

Best practice review of data use for
quality improvement

Maria Sibanyoni from the WRHI

reported on the implementation of a quality-improvement
intervention in Johannesburg. 8 The
intervention incorporated quality-improvement meetings with
staff, collaborative learning workshops, process mapping and a
data dashboard to improve initiation and adherence to ART. This

effort succeeded in creating an inter-facility referral
network and focused on data-driven processes that provided
clear and achievable targets for meeting client needs. These
achievements can be replicated in other locations.

Theunis Hurter, from Anova’s Cape Winelands project,
demystified TIER.net reporting for the audience.
TIER.net is being expanded into facilities throughout the
country (Fig. 2 shows its growing use in Johannesburg). In the
Winelands, TIER.net has helped clinicians and policy makers at
facilities and the district level to identify defaulters,
track and trace patients, and even identify PMTCT programme
gaps. Specific to PMTCT, Hurter and DoH colleagues in the Cape
Winelands identified, through the use of routine data, that
facilities in the district had initiated ART in more
under-2-year-olds than had been offered PMTCT services – a
clear service-delivery gap. Like this example, one key element
in using data for effective programme and data quality
improvement is the presence of facility managers who empower
their data capturers and others to give feedback on the data
and make note of any trends, issues or remarkable issues in
the data.

Fig. 2. Mapping TIER.net progress in Johannesburg, May 2012.
9

Existing data tools available to
facilities, clinicians and policy makers

Existing tools, organisations and
methodologies are in abundance, but greater coverage
and use of these tools is still needed. The DHIS, for example,
can be used to identify data quality issues through min/max out-of-range graphs and data completeness reports. The
Prevention of Mother-to-Child Transmission Action Framework

is effective for target-setting and monitoring programme
performance. As much of this information was new to the
conference audience, we suggest that raising awareness of
these tools is still necessary.

Mashudu Rampilo shared the results of an informal Data Quality
Audit comparing source documents (registers) to facility reports
and DHIS data specific to HIV testing, the PMTCT programme and
ART in Mopani, Limpopo Province. Although from a different
setting, the audience was both familiar and shocked with the
results. Rampilo’s results showed wide variation and regular
disagreement between each of the three data points (the source,
facility report and the DHIS). As noted in the DHMIS overview,
data accuracy is the responsibility of the staff at facility,
sub-district and district level.

One method to improve service delivery at the facility
level is treatment-gap modelling. This uses
baseline data, national targets and comparisons between people
receiving treatment and those eligible for treatment, to
estimate where the biggest gaps in service coverage exist, and
where more needs to be done to meet local, provincial and
national health indicator targets. This approach was adapted
from the work of Barker and Venter.10

The available, but under-utilised (as
remarked from conference attendees) DHB contains a
comprehensive set of indicators to inform planning at all
levels in the government and NGO sectors. Candy Day from the
Health Systems Trust highlighted how the DHB can be used to
provide an overall view of district health performance at the
primary healthcare level, and to provide comparative data to
monitor the overall quality of service within a district.

One final strategy for data use is the Three Tier ART
Monitoring and Evaluation (M&E) Strategy, of which the ART
M&E standard operating procedures (SOPs) are a key element.
Catherine White presented this tool, which is essential to
quality data collection and use of M&E of ART.

Recommendations

While facilitating the final discussion, Dr
Cephas Chikanda, Anova’s Head of Health Systems Strengthening,
and Prince Dulaze, Anova’s M&E co-ordinator for
Johannesburg, solicited participant feedback to consolidate the
key points that the audience had derived from the day’s
presentations. The participants’ recommendations included:

• There is a need
for better communication about the data within facilities
between clinicians, facility managers and data collectors, as
well as between the different levels of the health system. For
example, facility and district managers need to communicate what
the data tell them about service delivery and resources.

• Accountability
for the data is the responsibility of everyone, from facility
and district data collectors, to district managers and policy
makers at the national level. Accountability includes knowing
the data elements, what the data reveal about health service
delivery and outcomes, and how to accurately and efficiently use
data to improve the health system.

• The continuous
revision of data-collection tools and systems is a concern.
Standardisation of tools and systems according to the DHMIS
would facilitate correct and timely completion of collection
tools, assist users in becoming familiar and comfortable with
the data tools, and make it easier for users and collectors to
identify issues and errors. Standardisation is one way to
contribute to continuous quality improvement, as well as the
development and use of tools and strategies for the immediate-
and long-term.

• In order
for the health system to use data most efficiently for its
best effect, it is important to value good quality data as
central to quality healthcare provision and worthy of
investing time and resources. This includes
sharing the results of data collection and interpretation with
health services and the public. Data must also be prioritised
within the system to highlight its worth as a valuable tool to
improve health service delivery.

Acknowledgement.
The conference was funded by PEPFAR through the United States
Agency for International Development (USAID) under co-operative

agreement 674-A-00-08-00009-00 to the Anova Health Institute.
The opinions expressed herein are those of the authors and do
not necessarily reflect the views of USAID/PEPFAR.

References

1. National
Department of Health. District Health Management Information
System (DHMIS) Policy 2011. Pretoria: DoH, 2011.
http://www.doh.gov.za/docs/policy/2012/dhmis.pdf (accessed 1
October 2012).

2. National
Department of Health. The Aid Effectiveness Framework for
Health in South Africa. Pretoria: DoH, 2012.
http://www.doh.gov.za/docs/stratdocs/2012/aideffect.pdf
(accessed 1 October 2012).

3. President’s

Emergency Plan for AIDS Relief (PEPFAR). Partnership
Framework in Support of South Africa's National HIV &
AIDS and TB Response 2012/13 - 2016/17 between the
Government of the Republic of South Africa and the
Government of the United States of America. Washington:
PEPFAR, 2010.
http://www.pepfar.gov/countries/frameworks/southafrica/index.htm
(accessed 1 October 2012).

3. President’s

4. National
Department of Health. District Health Management Information
System (DHMIS) Policy 2011. Pretoria: DoH, 2011.
http://www.doh.gov.za/docs/policy/2012/dhmis.pdf (accessed 1
October 2012).

5. Garrib A,
Stoops N, McKenzie A, et al. An evaluation of the District
Health Information System in rural South Africa. S Afr Med J
2008;98(7):549-552.

6. Mate KS,
Bennett B, Mphatswe W, et al. Challenges for routine health
system data management in a large public programme to prevent
mother-to-child HIV transmission in South Africa. PLoS ONE
2009;4(5):e5483.
[http://dx.doi.org/10.1371/journal.pone.0005483]

7. National
Department of Health. National Core Standards for Health
Establishments in South Africa. Pretoria: DoH, 2011.
http://www.sarrahsouthafrica.org/LinkClick.aspx?fileticket=YnbSHfR8S6Q%3D&tabid=2327

(accessed 1 October 2012).

8. Webster PD,
Sibanyoni M, Malekutu D, et al. Using quality improvement to
accelerate highly active antiretroviral treatment coverage in
South Africa. BMJ Qual Saf 2012;21(4):315-324.
[http://dx.doi.org/10.1136/bmjqs-2011-000381]

9. MEASURE
Evaluation. Excel2GoogleEarth (E2G). Chapel Hill: MEASURE
Evaluation.
http://www.cpc.unc.edu/measure/tools/monitoring-evaluation-systems/e2g
(accessed 1 March 2012).

10. Barker PM,
Venter F. Setting district-based annual targets for HAART and
PMTCT: A first step in planning effective intervention for the
HIV/AIDS epidemic. S Afr Med J 2007;95:916-917.
http://www.ihi.org/knowledge/Pages/Tools/SouthAfricaHAARTCalculator.aspx
(accessed 1 April 2012).