Philipp J Otolaryngol Head Neck Surg 2011; 26 (1): 10-15 c Philippine Society of Otolaryngology – Head and Neck Surgery, Inc. PhiliPPine Journal of otolaryngology-head and neck Surgery Vol. 26 no. 1 January – June 2011 ORIGINAL ARTICLES 10 PhiliPPine Journal of otolaryngology-head and neck Surgery ABSTRACT Objectives: To calculate the accuracy, sensitivity, specificity and positive predictive values of the Siemens HearCheck™ Navigator in detecting hearing loss and to compare values of these parameters when the examination is done in a soundproof booth and in a quiet room. Methods: Design: Analytical, cross-sectional study Setting: Tertiary Public University Hospital Patients: Patients seen at the Ear Unit of a tertiary public university hospital from June 2009 to August 2010 were tested using the Siemens HearCheck™ Navigator and pure tone audiometry, inside a soundproof audiometry booth and in a quiet room with an ambient noise of 50dB, with a different investigator for each examination. Each ear was treated as a separate subject. Results obtained from the HearCheck™ Navigator were designated as observed values and were classified as “no hearing loss” for green light, and “with hearing loss” for yellow or red lights. Results were compared with pure tone air conduction averages designated as gold standard values. Normal hearing acuity (0-25 dB) was classified as no hearing loss. Pure tone air conduction averages of 26dB and above were classified as “with hearing loss” and were further stratified as mild hearing loss (26-40dB) and moderate or worse hearing loss (>41 dB). Observed and gold standard values were compared and tabulated in a 2x2 table for all levels of hearing loss, mild hearing loss, and moderate or worse hearing loss. Accuracy, sensitivity, specificity, positive and negative predictive values of the Siemens HearCheck™ Navigator inside a soundproof audiometry booth and in a quiet room were determined using pure tone audiometry as the gold standard. Results: 100 patients (200 ears) were tested, with a median age of 43 years old (range 15-75), and an almost equal number of male and female participants (52 males, 48 females). Accuracy rate of the Siemens HearCheck™ Navigator inside the soundproof audiometry booth and in a quiet room were 82.5% and 84% respectively for all levels of hearing loss. Sensitivity, specificity, positive and negative predictive values were similar whether the examination was done inside the soundproof audiometry booth or in a quiet room. These values were notably higher in Accuracy of Siemens HearCheck™ Navigator as a Screening Tool for Hearing Loss Kathleen R. Fellizar-Lopez, MD, MPH1 Generoso T. Abes, MD, MPH1,2 Ma. Rina T. Reyes-Quintos, MD, MClinAud1,2 Ma. Leah S. Tantoco, MD, MClinAud2 1Department of Otorhinolaryngology Philippine General Hospital University of the Philippines Manila 2Philippine National Ear Institute National Institutes of Health University of the Philippines Manila Correspondence: Dr. Kathleen R. Fellizar-Lopez Department of Otorhinolaryngology Ward 10, Philippine General Hospital Taft Avenue, Ermita, Manila 1000 Philippines Phone: (632) 554 8400 loc 2152 Fax: (632) 525 5444 Email: orlpgh@yahoo.com Reprints will not be available from the author The authors declared that this represents original material that is not being considered for publication or has not been published or accepted for publication elsewhere, in full or in part, in print or electronic media; that the manuscript has been read and approved by all the authors, that the requirements for authorship have been met by each author, and that each author believes that the manuscript represents honest work. Disclosures: The HearCheck™ Navigator was complementarily provided by Siemens for research purposes. The authors signed disclosures that there are no financial or other (including personal) relationships, intellectual passion, political or religious beliefs, and institutional affiliations that might lead to a conflict of interest. Presented at the Analytical Research Contest (1st Place), Philippine Society of Otolaryngology Head and Neck Surgery, Natrapharm Patriot Building, Paranaque City, Philippines,October 13, 2010 patients with moderate or worse hearing loss compared to patients with mild hearing loss. Conclusion: The Siemens HearCheck™ Navigator shows potential as an accurate, portable, easy-to-use tool to screen for hearing loss, especially for cases of moderate or worse hearing loss, without the need for soundproof audiometry booths or special training. It is recommended that further studies be done to differentiate degrees of hearing loss, and to evaluate its usefulness in other target populations, including school children and the elderly. Keywords: hearing screening, hearing screening tool, accuracy, hearing loss, HearCheck™ Navigator Hearing loss is a prevalent problem in adults and children. It can have long-term negative consequences, resulting in loss of productivity, social stigma and low self-esteem.1,2 Half of all cases of hearing impairment are avoidable through prevention, early diagnoses and management.3 Hearing screening is of utmost importance for early detection and intervention and in decreasing this functional disability.4 Like most developing countries, the Philippines is in need of a hearing screening instrument that is available, affordable, easy to use and cost-effective, that can be brought to remote communities to be used in the primary care setting by community health workers. This screening tool must be accurate and reliable. Bedside tests (ie. finger rub, ballpen click, watch tick, whispered speech, Rinne and Weber) and disability questionnaires, although easy to perform, are subjective, difficult to standardize and have suboptimal sensitivity (60%), relatively good specificity (74%), and variable positive predictive value (24-100%) for detecting hearing loss.5 In a local study, the 512Hz tuning fork was shown to be an accurate and precise hearing screening tool, with a high accuracy rate of 97%, specificity of 97%, sensitivity of 91% and a positive predictive value of 81%.6 However, tuning fork tests, while inexpensive, require proper training, particularly the proper movement for the production of sound and placement of the tool in relation to the external auditory meatus, and interpretation of results. A collaborative program between the Philippine National Ear Institute and the Department of Education included training of school nurses in hearing screening with the use of a penlight and a 512Hz tuning fork.7 Among the observed difficulties with the use of the tuning fork was difficulty in eliciting sound and/or vibration because of the inability to perform the proper wrist movement, particularly in the presence of arthritic joints, as well as wrist pain from contact with the tuning fork. A study by Burkey et al. in 1998 showed that the Rinne tuning fork test could be very effective at detecting hearing loss when performed by an experienced examiner and when masking was employed. Sensitivity was lower when masking was not used and lowest when the Rinne test was performed by a less-experienced investigator. In the primary care setting, the Rinne test would be an effective part of a screening program for conductive hearing loss, but should not be the sole indicator for referral to a hearing center for other examinations.8 Objective examinations, such as otoacoustic emissions (OAEs), auditory brainstem response (ABR) and auditory steady state response (ASSR) are currently gaining popularity and have allowed early physiologic detection of hearing impairment. Prospects for this trend in developing countries remain doubtful because of adverse socio- economic conditions9 either leading to lack of equipment or poor access to facilities. Otoacoustic emissions and pure tone audiometry are the recommended equipment in hearing centers today. These instruments, along with ABR and ASSR, require expensive and delicate equipment which are difficult to transport, technical know-how and proper training on handling and operation, rely on electricity and are not readily available and accessible. Because of the paucity of these machines, there is still a lack of knowledge on the prevalence of hearing loss in the country. An integrated national hearing screening program has yet to be developed. While searching for an alternative tool for hearing screening, the investigators considered portable handheld screening audiometers which are available in the country today. One such audiometer is the Siemens HearCheck™ Navigator (Siemens, Germany). It is a simple and portable instrument. However, there has been no study which measures its accuracy and reliability based on a PubMED MEDLINE literature search (search words: Siemens HearCheck Navigator, hearing screening, accuracy, reliability). This paper aims to determine the accuracy of the Siemens HearCheck™ Navigator in screening for hearing loss using pure tone audiometry diagnosis as gold standard. Specifically, it aims: 1. To calculate the accuracy, sensitivity, specificity and positive predictive values of the Siemens HearCheck™ Navigator in detecting hearing loss; 2. To compare values of the above parameters when the examination is done in a soundproof booth and in a quiet room; 3. To determine accuracy, sensitivity, specificity and positive predictive value among patients with mild hearing loss only (on pure tone audiometry); and 4. To determine accuracy, sensitivity, specificity and positive predictive value among patients with at least moderate hearing loss (on pure tone audiometry). PhiliPPine Journal of otolaryngology-head and neck Surgery Vol. 26 no. 1 January – June 2011 ORIGINAL ARTICLES PhiliPPine Journal of otolaryngology-head and neck Surgery 11 METHODS Patients referred to the Ear Unit of a tertiary government hospital for pure tone audiometry from June 2009 to August 2010 were included in this analytical cross-sectional study. Informed consent was obtained and demographic data for each patient was taken. No harm was done to any of the study participants. The study was conducted in accordance to the principles of the Declaration of Helsinski. The Siemens HearCheck™ Navigator is a portable handheld screening audiometer, with three light indicators (green, yellow, red), a start button, battery compartment housing two AAA batteries, ear cover, label and a disposable ear cup. (Figure 1) It has been factory calibrated and should be recalibrated three years from the manufacturing date. It presents sound at 35dB, 55dB and 75dB at test frequencies of 375Hz, 1000Hz and 3000Hz. All patients were tested using the device in a quiet room, followed PhiliPPine Journal of otolaryngology-head and neck Surgery Vol. 26 no. 1 January – June 2011 ORIGINAL ARTICLES 12 PhiliPPine Journal of otolaryngology-head and neck Surgery by a test done inside a soundproof audiometry booth, and pure tone audiometry. (Figure 2) Five-minute rest periods were allotted between each test. Each ear was treated as a separate subject. 1. HearCheck™ Navigator testing Physician A performed the test with the device in a quiet room. The ambient noise inside the room (average of 50dB) was determined using a TES1350A sound level meter (TES Electrical Electronic Corp., Taiwan, 2000) prior to each test. The device was held gently to the head of the patient, with the ear cover completely surrounding the ear ensuring skin contact all around. The test sequence started when the start button was pressed. A short automatic functional test was performed, afterwhich all three light indicators would flash, indicating that the device was ready to start. The patient was instructed to raise his/her hand when a tone was heard. (Figure 2) When the patient raised his/her hand, the examiner pressed the start button. The test for the particular frequency was terminated automatically when the start button was not pressed within 20 seconds. The result of the test was indicated with a red, yellow or green light for another 20 seconds. Physician B performed the test with the device in a soundproof audiometry booth, with the same preparations and steps as described above. Each examiner was blinded to the results of the other examinations. 2. Pure tone testing with a diagnostic audiometer A technician blinded to the results of the two previous tests performed all the pure tone audiometry tests using an AD229b Diagnostic Audiometer (Interacoustics A, S, Assess, Denmark, 2005) that was calibrated weekly. Figure 1. Siemens HearCheck™ Navigator portable handheld screening audiometer with disposable ear cups. Table 1. World Health Organization Grades of Hearing Impairment (Geneva, 1991)10 Grade of Impairment Corresponding audiometric ISO value (Average of 500, 1000 and 2000Hz) None Mild impairment Moderate impairment Severe impairment Profound impairment including deafness 25dB or better 26-40 dB 41-60 dB 61-80 dB 81dB and above Table 2. Two-by-two table HearCheck™ Navigator Pure tone audiometer With hearing loss (+) No hearing loss (-) No hearing loss (-) False positive True negative With hearing loss (+) True positive False negative Figure 2. Photographs illustrating test sequence using the portable handheld hearing screening instrument by Physician A in a quiet room (a) and Physician B inside a soundproof booth (b). PhiliPPine Journal of otolaryngology-head and neck Surgery Vol. 26 no. 1 January – June 2011 ORIGINAL ARTICLES PhiliPPine Journal of otolaryngology-head and neck Surgery 13 Results obtained from the HearCheck™ Navigator were designated as observed values and were classified as positive for hearing loss with yellow or red results, and negative for hearing loss with green results. The results were compared with pure tone air conduction average (500Hz, 1000kHz, 2000kHz), which were designated as gold standard values. Normal hearing acuity (0-25 dB) was classified as no hearing loss. Pure tone air conduction averages of 26dB and above were classified as positive for hearing loss, based on the World Health Organization Grades of Hearing Impairment, and were further stratified into mild hearing loss (26-40dB hearing loss) and moderate or worse hearing loss (>41dB hearing loss)10 (Table 1). Observed and gold standard values were compared according to presence or absence of hearing loss. The results were tabulated in a two by two table (Table 2). Accuracy, sensitivity, specificity, positive and negative predictive values of the Siemens HearCheck™ Navigator inside the soundproof audiometry booth and in a quiet room were determined, using pure tone audiometry as the gold standard. Computations were done for all patients, patients with mild hearing loss only and those with at least moderate hearing loss Computed values of results obtained inside the soundproof audiometry booth and in a quiet room were compared using the Z test for two proportions, with a level of significance of 0.05. RESULTS A total of 200 ears (100 patients) were tested, with a median age of 43 years old (range 15-75 years old), with almost an equal number of male and female participants (52 males, 48 females, ratio of 1.08:1). The values obtained from 200 ears are shown in Table 3 for all levels of hearing loss, Table 4 for mild hearing loss and Table 5 for moderate or worse hearing loss. The corresponding computations obtained from the results of each of the above tables are shown in Table 6. No statistically significant difference was noted (α=0.05) whether the examination was done inside a soundproof audiometry booth or in a quiet room for all computed values. Patients with hearing loss were grouped into those with mild hearing loss only and those with at least moderate hearing loss. No statistically significant difference was noted in the computed values for both groups inside the soundproof booth and in a quiet room. Note, however, that accuracy and sensitivity were significantly lower for patients with mild hearing loss compared to patients with at least moderate hearing loss. Sensitivity of the HearCheck™ Navigator was high in patients from the latter group but was suboptimal in patients from the former. Table 3. HearCheck Navigator vs Puretone Audiometry for all levels of hearing loss HearCheck™ Navigator Pure tone audiometer With hearing loss Without hearing loss Total Inside booth Quiet roomTotal Total With hearing loss 104 30 134 With hearing loss 105 29 134 Without hearing loss 5 61 66 Without hearing loss 3 63 66 109 91 200 108 92 200 Table 4. HearCheck™ Navigator vs Puretone Audiometry for mild hearing loss HearCheck™ Navigator Pure tone audiometer With hearing loss Without hearing loss Total Inside booth Quiet roomTotal Total With hearing loss 19 17 36 With hearing loss 16 20 36 Without hearing loss 5 61 66 Without hearing loss 3 63 66 24 78 102 19 83 102 Table 5. HearCheck™ Navigator vs Puretone Audiometry for moderate and worse hearing loss HearCheck™ Navigator Pure tone audiometer With hearing loss Without hearing loss Total Inside booth Quiet roomTotal Total With hearing loss 85 13 98 With hearing loss 89 9 98 Without hearing loss 5 61 66 Without hearing loss 3 63 66 90 74 164 92 72 164 Table 6. Results obtained for the different values inside the booth and in a quiet room for all levels of hearing loss, mild hearing loss and moderate or worse hearing loss All levels of hearing loss Mild hearing loss (26-40dB) Moderate or worse hearing loss (>41dB) Parameter Accuracy Sensitivity Specificity Positive predictive value Negative predictive value Inside booth (%) 82.5 77.61 92.42 95.41 67.03 Inside booth (%) 78.43 52.77 92.42 79.17 78.21 Inside booth (%) 89.02 86.73 92.42 94.44 82.44 Quiet room (%) 84 78.36 95.45 97.22 68.48 Quiet room (%) 77.45 44.44 95.45 84.21 75.90 Quiet room (%) 92.68 90.92 95.45 96.74 87.50 z-test* NS NS NS NS NS z-test NS NS NS NS NS z-test NS NS NS NS NS PhiliPPine Journal of otolaryngology-head and neck Surgery Vol. 26 no. 1 January – June 2011 ORIGINAL ARTICLES 14 PhiliPPine Journal of otolaryngology-head and neck Surgery Table 7. Summary of portable handheld screening audiometer studies Study No. of Patients Accuracy YearType of handheld screen- ing audiometer used Penaflor, et al6 Lichtenstein, et al12 Ciurlia-Guy, et al13 125 178 104 94% 78.6% 88.17% 2001 1998 1993 Siemens SD 10 handheld tester Audioscope Audioscope DISCUSSION With puretone audiometry as the gold standard, the high specificity rate of the HearCheck™ Navigator, indicates that we can rule in the presence of hearing loss when the device shows a yellow or red light, whether the examination is done inside the soundproof booth (92.42%) or in a quiet room (95.45%). The high positive predictive value means that among those with hearing loss based on the HearCheck™ Navigator results, the probability of having actual hearing loss is 9 out of 10, especially in cases of moderate or worse hearing loss. False positive results are practically nil. However, sensitivity of the HearCheck™ Navigator is not as high as its specificity. For all types of hearing loss combined, sensitivity is only 77.61% in a soundproof booth and 78.36% in a quiet room, indicating that only 7 out of 10 of those with hearing loss according to the device have actual hearing loss, and that hearing loss can be missed. The negative predictive value of the device is likewise not very high for all levels of hearing loss. 4 out of 10 may be mislabeled as normal when in fact they may indeed have hearing loss. Results obtained when computations are done separately for those with mild hearing loss and moderate or worse hearing are higher for the latter. Accuracy and sensitivity of the HearCheck™ Navigator were noted to be significantly higher in patients with at least moderate degrees of hearing loss compared to patients with mild hearing loss. While several cases of mild hearing loss were correctly labeled by the device as having hearing loss (yellow light), most of the false negatives had mild hearing loss on pure tone audiometry. This indicates that the HearCheck™ Navigator is better at detecting moderate or worse hearing loss (41dB and above air conduction average) than mild levels of hearing loss (26 to 40dB air conduction average), and that mild hearing loss may be missed. Thus, with these current values, we cannot confidently rule out hearing loss in the presence of a negative test as indicated by a green light but it can be confidently ruled in by the presence of a red or yellow light. Caution must be employed in observing patients that we suspect of having hearing loss but register a negative result on the HearCheck™ Navigator, so as not to miss a hearing problem. In such a case, the test may be repeated in order to validate a previous negative result. Referral for further testing (ie. pure tone audiometry, ABR/ASSR) may be indicated if screening results are negative but hearing loss is highly suspected. According to Cadman et al., a high sensitivity rate, high specificity rate and high positive predictive values are attributes of a good test for a screening program.11 The HearCheck™ Navigator was able to fulfill two of these characteristics – a high specificity rate and a high positive predictive value. It can confidently rule in hearing loss, but we cannot say that the device could confidently rule it out. In cases of moderate or worse hearing loss, all these criteria were fulfilled by the HearCheck™ Navigator. The accuracy rate of the HearCheck™ Navigator inside a soundproof booth or in a quiet room is 82.5% and 84% respectively. The accuracy rate of the HearCheck™ Navigator is comparable to other portable handheld screening audiometers used in other studies. 6,12-13 (Table 7) These studies had bigger sample sizes compared to this. Hence, this study is being continued to obtain a more adequate sample size for a more significant analysis. It is important to note also that the devices used in these studies are expensive, require technical know-how and special training in their operation and interpretation of results, and cannot be readily brought to remote communities to be used by local community health workers. In contrast to the 512Hz tuning fork, whose usefulness as a screening tool may be limited to disorders presenting with low-frequency hearing loss (ie. external or middle ear disorders),6 the frequencies presented by the HearCheck™ Navigator include higher frequencies. Thus, this device is capable of detecting disorders presenting as high frequency hearing loss, such as early presbycusis. This study was conducted in an ideal test environment. Subjects included were referrals from physicians, hence, the positive predictive value could be overestimated as disease prevalence is known to be higher in referral centers than in the community or in the primary care setting.11 It is recommended that this test be validated by studies done among different populations at the community level, including the elderly and the pediatric population. The HearCheck™ Navigator is a portable, light-weight, non-invasive device that is very simple and easy to use. No statistically significant difference was noted in the values obtained whether the test was done inside a soundproof audiometry booth or in a quiet room indicating that a soundproof booth is not necessary for the device to perform its function. It does not rely on an external electric supply and runs on two AAA batteries. Results are available in seconds. No intensive PhiliPPine Journal of otolaryngology-head and neck Surgery Vol. 26 no. 1 January – June 2011 ORIGINAL ARTICLES PhiliPPine Journal of otolaryngology-head and neck Surgery 15 ACKNOWLEDGEMENTS We acknowledge Romeo L. Villarta, MD, MPH, DrPH (Cand) and Erasmo Gonzalo DV Llanes, MD for their scientific advice, Marieflor Cristy M. Garcia, MD, Kathryn Llanos Dee, MD and Ma. Luz M. San Agustin BSBA for the data collection, Hypte Raymund V. Aujero, MD for the photographs and Ms. Michelle Mañanes for general assistance with this research. REFERENCES 1. Mulrow CD, Aguilar C, Endicott JE, Tuley MR, Velez R, Charlip WS, et al. Quality-of-life changes and hearing impairment. A randomized trial. Ann Intern Med. 1990 Aug 1; 113(3):188-194. 2. Yueh B, Shapiro N, Maclean CH, Sheckelle PG. Screening and management of adult hearing loss in primary care. JAMA. 2003 Apr 16; 289:1976-85. 3. WHO Deafness and hearing impairment fact sheet N°300. Place: World Health Organization. April 2010. [Cited 2010 May 26] Available from:http://www.who.int/mediacentre/factsheets/ fs300/en/index.html. 4. Wang YF, Wang SS, Tai CC, Lin LC, Shiao AS. Hearing screening with portable screening pure- tone audiometer and distortion-product otoacoustic emissions. Zhonghua Yi Xue Za Zhi (Taipei). 2002; 65(6); 285-92. 5. Boatman DF, Miglioretti DL, Eberwein C, Alidoost MS, Reich SG. How accurate are bedside hearing tests? Neurology. 2007 Apr 17; 68(16):1311-4. 6. Penaflor NA, Hilario O, Abes GT. Accuracy of 512 Hertz Air Conduction Test for Hearing Screening. Philipp J Otolaryngol Head Neck Surg. 2001; 16(1):25-33. 7. Gloria-Cruz TI, Chiong CM, Chan AL, Llanes ED, Reyes-Quintos RT, Abes GT. Training of nurses in ear examination and hearing screening in the school setting. Southeast Asian J Trop Med Pub Health. 2007 Jan;38(1):188-94. 8. Burjey JM, Lippy WH, Schuring AG, Rizer FM. Clinical utility of the 512-Hz Rinne tuning form test. Am J Otol. 1998 Jan; 19(1):59-62. 9. Olusanva B. Early detection of hearing impairment in a developing country: what options? Audiology. 2001 May-Jun;40(3):141-7. 10. WHO Grades of Hearing Impairment. [Monograph? on the internet] Geneva; World Health Organization; 1991. [Cited 2010 May 26]. Available from http://www.who.int/pbd/deafness/ hearing_impairment_grades/en/index.html. 11. Cadman D, Chambers L, Feldmann W, Sackett D. Assessing the effectiveness of community screening program. JAMA. 1984 Mar 23-30; 251(12): 1580-1585. 12. Lichtenstein MJ, Bess FH, Logan SA. Validation of screening tools for identifying hearing- impaired elderly in primary care. JAMA. 1988 May 20; 259(19): 2875-2878. 13. Ciurlia-Guy E, Cashman M, Lewsen B. Identifying hearing loss and hearing handicap among chronic care elderly people. Gerontologist. 1993;33:644-649. training is required for its operation. As such, it can be easily used by local community workers, non-physicians and non-audiologists. It can be brought to communities who have no access to hearing screening centers. Since the ultimate utility of this study entails its use by local community examiners, the results should also be validated by studies performed among different populations carried out by actual local community health workers. Corollary to this, inter-investigator variability must also be evaluated in order to validate if it can be easily used by different investigators in the community but still arrive at the same results in a patient. The use of the Siemens HearCheck™ Navigator as a tool for a screening program appears to be promising. While the device appears to be ideal for community use based on its physical characteristics, its accuracy and reliability as a screening tool must further be validated.