ISDS Annual Conference Proceedings 2017. This is an Open Access article distributed under the terms of the Creative Commons Attribution- Noncommercial 3.0 Unported License (http://creativecommons.org/licenses/by-nc/3.0/), permitting all non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. ISDS 2016 Conference Abstracts Utility of Natural Language Processing for Clinical Quality Measures Reporting Dino P. Rumoro1, Shital C. Shah*1, Gillian S. Gibbs1, Marilyn M. Hallock1, Gordon M. Trenholme1 and Michael J. Waddell2 1Emergency Medicine, Rush University Medical Center, Chicago, IL, USA; 2Pangaea Information Technologies, Chicago, IL, USA Objective To explain the utility of using an automated syndromic surveillance program with advanced natural language processing (NLP) to improve clinical quality measures reporting for influenza immunization. Introduction Clinical quality measures (CQMs) are tools that help measure and track the quality of health care services. Measuring and reporting CQMs helps to ensure that our health care system is delivering effective, safe, efficient, patient-centered, equitable, and timely care. The CQM for influenza immunization measures the percentage of patients aged 6 months and older seen for a visit between October 1 and March 31 who received (or reports previous receipt of) an influenza immunization. Centers for Disease Control and Prevention recommends that everyone 6 months of age and older receive an influenza immunization every season, which can reduce influenza- related morbidity and mortality and hospitalizations. Methods Patients at a large academic medical center who had a visit to an affiliated outpatient clinic during June 1 - 8, 2016 were initially identified using their electronic medical record (EMR). The 2,543 patients who were selected did not have documentation of influenza immunization in a discrete field of the EMR. All free text notes for these patients between August 1, 2015 and March 31, 2016 were retrieved and analyzed using the sophisticated NLP built within Geographic Utilization of Artificial Intelligence in Real-Time for Disease Identification and Alert Notification (GUARDIAN) – a syndromic surveillance program – to identify any mention of influenza immunization. The goal was to identify additional cases that met the CQM measure for influenza immunization and to distinguish documented exceptions. The patients with influenza immunization mentioned were further categorized by GUARDIAN NLP into Received, Recommended, Refused, Allergic, and Unavailable. If more than one category was applicable for a patient, they were independently counted in their respective categories. A descriptive analysis was conducted, along with manual review of a sample of cases per each category. Results For the 2,543 patients who did not have influenza immunization documentation in a discrete field of the EMR, a total of 78,642 free text notes were processed using GUARDIAN. Four hundred fifty three (17.8%) patients had some mention of influenza immunization within the notes, which could potentially be utilized to meet the CQM influenza immunization requirement. Twenty two percent (n=101) of patients mentioned already having received the immunization while 34.7% (n=157) patients refused it during the study time frame. There were 27 patients with the mention of influenza immunization, who could not be differentiated into a specific category. The number of patients placed into a single category of influenza immunization was 351 (77.5%), while 75 (16.6%) were classified into more than one category. See Table 1. Conclusions Using GUARDIAN’s NLP can identify additional patients who may meet the CQM measure for influenza immunization or who may be exempt. This tool can be used to improve CQM reporting and improve overall influenza immunization coverage by using it to alert providers. Next steps involve further refinement of influenza immunization categories, automating the process of using the NLP to identify and report additional cases, as well as using the NLP for other CQMs. Table 1. Categorization of influenza immunization documentation within free text notes of 453 patients using NLP Keywords GUARDIAN; clinical quality measures; Influenza immunization; Natural language processing Acknowledgments GUARDIAN is funded by the US Department of Defense, Telemedicine and Advanced Technology Research Center, Award numbers W81XWH-09-1-0662 and W81XWH-11-1-0711. *Shital C. Shah E-mail: Shital_Shah@rush.edu Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 9(1):e27, 2017