Layout 1


ISDS Annual Conference Proceedings 2012. This is an Open Access article distributed under the terms of the Creative Commons Attribution-
Noncommercial 3.0 Unported License (http://creativecommons.org/licenses/by-nc/3.0/), permitting all non-commercial use, distribution, and
reproduction in any medium, provided the original work is properly cited.

ISDS 2012 Conference Abstracts

Automated Surveillance of Outpatients with Pneumonia:
A Performance Evaluation
Hongzhang Zheng1, 2, Tariq Siddiqui1, 2 and Sylvain DeLisle*1, 2

1VA Maryland Health Care System, Baltimore, MD, USA; 2Medicine, University of Maryland, Baltimore, MD, USA

Objective
To determine if influenza surveillance should target all patients

with acute respiratory infections (ARI) or only track pneumonia
cases.

Introduction
Effective responses to epidemics of infectious diseases hinge not

only on early outbreak detection, but also on an assessment of disease
severity. In recent work, we combined previously developed ARI
case-detection algorithms (CDA) [1] with text analyses of chest im-
aging reports to identify ARI patients whose providers thought had
pneumonia. In this work, we asked if a surveillance system aimed at
patients with pneumonia would outperform one that monitors the full
severity spectrum of ARI.

Methods
Time series of daily casecounts (backgrounds) were created by ap-

plying either an ARI CDA (ARI ICD-9 codeset [1]) or a Pneumonia
CDA (ARI ICD-9 codes AND chest imaging obtained AND positive
results from automated text analysis that identify those chest imaging
reports that support the diagnosis of pneumonia) to electronic med-
ical record (EMR) entries related to outpatient encounters at the VA
Maryland Health Care System. We used an age-structured metapop-
ulation influenza epidemic model for Baltimore to inject factitious
influenza cases into backgrounds. Injections were discounted by the
known sensitivity of the ARI CDA [1]. For injections into the pneu-
monia backgrounds time series, factitious ARI cases were further dis-
counted by the expected pneumonia rate in the modeled influenza
epidemic (10%). From the time of injection, EARS or CUSUM sta-
tistics [2,3] were applied on each successive day on paired back-
ground+injection vs. background-only time series. Each
injection-prospective-surveillance cycle was repeated 52 times, each
time with the injection shifted to a different week of the one-year
study period (2010-11). We computed: 1) the “Detection Delay”, the
average time from injection to the first alarm present in the back-
ground+injection dataset but absent from the background-only
dataset; 2) the “False-Alarm Rate” (FAR), defined as the number of
unique false-alarms originating in the background-only dataset dur-
ing the study year, divided by 365 days. To create activity monitor-
ing operating characteristic (AMOC) curves, we empirically
determined the corresponding Delay-FAR pairs over a wide range of
alarm thresholds.

Results
The Figure compares AMOC curves for otherwise identical sur-

veillance systems that included either any ARI outpatient visits (red
circles, using the EARS W2c statistic [2] and blue triangles, using
the CUSUM statistic) or pneumonia (blue triangles, using the
CUSUM statistic modified for sparse data [3]). Note that Detection
Delay (y-axis) is lower at any given FAR when surveillance aims at
patients with pneumonia. Sensitivity analysis suggests that this ad-
vantage remains true when pneumonia complicates influenza ! 5% of
the time.

Conclusions
Our results suggest that EMR-based influenza surveillance that tar-

gets patients with pneumonia can outperform systems that monitor
all ARI patients.

Keywords
influenza; surveillance; modeling; simulations

References

[1] DeLisle S, South B, Anthony JA, Kalp E, Gundlapalli A, et al. Com-
bining Free Text and Structured Electronic Medical Record Entries
to Detect Acute Respiratory Infections. PLoS ONE (2010) 5(10):
e13377

[2] Hutwagner L, Thompson W, Seeman GM, Treadwell T. The bioter-
rorism preparedness and response early aberration reporting system
(EARS). Journal of Urban Health: Bulletin of the New York Acad-
emy of Medicine 2003; 80(2) (Suppl. 1) i89-i96.

[3] He, S., Huang, W. and Woodall, W. H. CUSUM charts for monitoring
a zero-inflated Poisson process. Qual. Reliab. Engng. Int. (2012) 28:
181–192.

*Sylvain DeLisle
E-mail: sdelisle@umaryland.edu

Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(1):e38, 2013