Hrev_master [Emergency Care Journal 2015; 11:5276] [page 43] Reliability and factor structure of brief emergency department patient satisfaction scale Maryam Atari,1,2 Saeed Akbari-Zardkhaneh,3 Mohammad Atari4 1Faculty of Management, University of Tehran, Tehran; 2Quality Improvement Department, Moheb Hospital, Tehran; 3Department of Psychology, Shahid Beheshti University, Tehran; 4Department of Psychology, University of Tehran, Tehran, Iran Abstract Patient satisfaction (PS) is an important factor for both patients and healthcare profes- sionals. Psychometrically sound assessment of PS is of absolute importance for quality improvement purposes particularly in private hospitals. One of the PS instruments with high reliability and validity is the Brief Emergency Department Patient Satisfaction Scale (BEPSS). This study aimed to investigate the factor structure of BEPSS in a private hospital. A total of 270 emergency patients from a pri- vate hospital filled the questionnaires. Confirmatory factor analysis was used to investigate the factor structure of BEPSS. General-factor and five-factor models of the instrument were compared. Internal consis- tency of the scale was evaluated using Cronbach’s alpha coefficients. The five-factor solution of the BEPSS had higher indices of fit and was psychometrically more appropriate. The factor structure was consistent with the original solution. All subscales were internally consistent. Cronbach’s alphas ranged between 0.59 and 0.88 for the five subscales. As a result, BEPSS is a valid and reliable instrument in order to be used in private hospitals and clin- ics. It may serve as a regular PS evaluation tool, which assesses five domains of PS in emergency rooms or for research purposes. Introduction Patient satisfaction (PS) may be considered as an indicator of quality in healthcare servic- es which is perceived by patients and is a com- plex concept in nature.1 Many factors should be correctly taken into account in order to make an appropriate condition for development and improvement of patient satisfaction particular- ly in hospitals.2 Satisfied patients show statis- tically higher adherence to their medical instructions, that is, satisfaction may be an important component in promoting health and psychological well-being in the society.3-5 There has been a proliferation of studies on PS over the last few decades.6 Although the field of emergency medicine is comparatively new, it has not been neglected in PS research. Yet, many of the existing Emergency Department (ED) studies concerning PS have serious methodological flaws, which has led to inconsistent and, sometimes, contradictory conclusions.7 Many studies have utilized psy- chometric instruments in order to measure PS;8-12 however, psychometric properties of such instruments have not been discussed at length13 and that is particularly important as emergency patients may have complex psy- chosocial issues as well as medical problems.14 A newly developed instrument15 has made an effort to overcome the issues in measurement of PS in ED settings. Brief Emergency Department Patient Satisfaction Scale (BEPSS) was developed pri- marily to overcome the aforementioned issues.15 Satisfactory psychometric properties of BEPSS were reported in its developmental levels. BEPSS has been reported to be a psy- chometrically sound instrument measuring five domains by 20 items in ED settings. These domains are named as emergency department staff (EDS), emergency department environ- ment (EDE), physician care satisfaction (PCS), general patient satisfaction (GPS), and patient’s family’s satisfaction (PFS). The utility of BEPSS may be beneficial for several reasons. First, it has been developed using comprehensive psychometric methods. Second, it covers all domains of patient satis- faction in ED settings. Using this scale could particularly benefit the domestic research as utilizing foreign instruments without reliabili- ty and validity could substantially impact the results. While reliability and validity of the scale were reported high, no study has investigated the factor structure of BEPSS using confirma- tory factor analysis (CFA). The present study aimed to investigate the factor structure of BEPSS using confirmatory factor analytic tech- niques. Materials and Methods Participants A total of 270 ED patients were recruited using convenience-sampling method from a private hospital in Tehran, Iran with monthly visits of 600 to 900 ED patients. All participants were admitted in either morning (7 a.m. - 3 p.m.) or evening (3 p.m. - 11 p.m.) shifts. Demographic characteristics of participants are summarized in Table 1. Measures Participants completed the BEPSS,15 a 20- item PS instrument which measures five major aspects of patient satisfaction among emergency patients (Appendix). Subscales are named as emergency department staff (EDS), emergency department environment (EDE), physician care satisfaction (PCS), general patient satisfaction (GPS), and patient’s fami- ly’s satisfaction (PFS). Response option was provided in a 4-point Likert scale ranging from completely disagree to completely agree. Total score of the scale may range between 20 and 80. Participants also provided their demo- graphic details consisting of gender, age, edu- cational background, and waiting time in the emergency room. Procedure Participants were patients from a private hospital. They were approached by a research assistant in the emergency room only after they had finished visiting the physician. Upon agreement to participate, they provided oral informed consent and completed a paper-and- pencil survey. Self-report questionnaires were completed on arrival and before discharge. Ethics committee of the hospital approved this study. All data were treated confidentially. According to the research assistant, the mean time to complete the survey was roughly 3-5 Emergency Care Journal 2015; volume 11:5276 Correspondence: Mohammad Atari, Department of Psychology, University of Tehran, Faculty of Psychology and Educational Sciences, Tehran, Iran. Tel. +98.912.6363263 - Fax. +98.21.88364403. E-mail: Atari@ut.ac.ir Key words: Patient satisfaction; Reliability; Validity; Emergency department; Factor analysis. Contributions: MaA: conception, design of the work, data collection, critical revising, final approval of the manuscript; SAZ: interpretation of the data, data analysis, research design, final approval of the manuscript; MoA: statistical analysis, research design, data collection, draft- ing the manuscript, final approval of the manu- script. Conflict of interest: the authors declare no poten- tial conflict of interest. Received for publication: 19 May 2015 Revision received: 24 July 2015. Accepted for publication: 30 July 2015. This work is licensed under a Creative Commons Attribution 3.0 License (by-nc 3.0). ©Copyright M. Atari et al., 2015 Licensee PAGEPress, Italy Emergency Care Journal 2015; 11:5276 doi:10.4081/ecj.2015.5276 Non co mmerc ial us e o nly [page 44] [Emergency Care Journal 2015; 11:5276] minutes. Participants were not remunerated and participation was on a voluntary basis. Statistical analysis Confirmatory factor analysis (CFA) was per- formed in this study. General-factor solution was compared to 5-factor model as suggested by scale developers.15 Various indices were compared in order to compare the goodness-of- fit of the two models. Finally internal consis- tency of each subscale was assessed using Cronbach’s alpha. Statistical analyses were performed using SPSS 22 and AMOS 19. In evaluating the goodness of fit to the data we report the model chi-square statistic asso- ciated with the p value, the comparative fit index (CFI), and the root mean square error of approximation (RMSEA). A non-significant value of the chi-square statistic indicates a good fit, however the test is sensitive to sam- ple size and should be considered in relation to its degrees of freedom (i.e., dividing chi- square value by its degrees of freedom should result in a value below 2, indicating a good model).16 Hu and Bentler’s combinational fit criteria for CFI and RMSEA indices17 were used. CFI equal to or superior to .95 and RMSEA less than .05 are considered to indicate a good fit. CFI equal to or superior to .90 and RMSEA less than .08 are considered to indicate a moderate but acceptable fit. Results The proposed model includes five latent variables. These were EDS (6 items), EDE (3 items), PCS (4 items), GPS (5 items), and PFS (2 items). The general-factor model was not fit; however, the 5-factor model showed higher indices for goodness-of-fit. The comparison of alternative models is presented in Table 2. The visual representation of the 5-factor model underlying BEPSS is presented in Figure 1. Cronbach’s alpha showed adequate values (i.e., >0.70) for each subscale except a margin- ally acceptable value for PFS. Results of relia- bility of the five subscales are presented in Table 3. Moreover, the overall alpha of the scale was high (alpha=0.94). Discussion Several problems have been identified as inherent in the analysis of PS in emergency departments. Definition of satisfaction, meth- ods of quantifying satisfaction, and large pop- ulation of patients in ED settings are consid- ered to be main issues in measuring PS.18 The BEPSS has been developed to overcome limita- tions of previous measures of PS in ED set- tings. The present study aimed to investigate the underlying factor structure of BEPSS. Based on the findings, the BEPSS seems to be a useful instrument for measurement of patient satisfaction in ED settings. The con- ducted analysis confirmed the superiority of the original five-factor solution for the present 20-item instrument. Confirmatory factor analysis was performed in order to compare two competing models. The first model posited that ED patient satis- faction was one-dimensional; while the second model was based on the original factor struc- ture of BEPSS.15 Results of CFA confirmed the original factor structure in the present sample. All fit indices fell within acceptable range, sup- porting the 5-factor solution for the instru- ment. Therefore, ED patient satisfaction con- sists of five domains as measured by BEPSS. The five subscales were found to be inter- nally consistent providing support for the reli- ability of the instrument; however, the fifth subscale (PFS) showed relatively low internal consistency. This may be explained by the fact that PFS consists of only two items. Fewer items in a subscale increase the possibility of lowered alpha coefficient. Adding few concep- tually similar items to this subscale would increase the internal consistency of the sub- scale. Items of this instrument are inclusive of contents of many complaints in emergency departments. A recent study, investigated the reasons of dissatisfaction among Iranian ED patients.19 Findings suggest that BEPSS’ items can represent those complaints (e.g. dissatis- Article Table 1. Demographic characteristics of participants. Variable N % Gender Male 167 61.86 Female 100 37.03 Missing 3 1.11 Waiting time (min) <5 min 143 52.96 5-10 min 29 10.74 >10 min 17 6.30 Missing 81 30.00 Educational background Lower than high school 36 13.33 High school 66 24.44 Associate’s degree 30 11.11 Bachelor’s 110 40.74 Master’s or higher 19 7.03 Missing 9 3.35 Admission time Morning shift 175 64.81 Evening shift 95 35.19 Night shift 0 0 Patient follow-up Hospital admission 71 26.30 Discharge 90 33.33 Deceased 11 4.07 Missing 98 36.30 Mean age, years (SD) 53.01 (17.81) Table 2. Confirmatory factor analysis of two structural models (N=270). Model χ2 χ2/df RMR GFI CFI PCFI RMSEA g-factor 911.26 5.36 0.02 0.73 0.79 0.71 0.13 5-factor 476.03 2.98 0.01 0.85 0.91 0.77 0.09 RMR, root mean square residual; GFI, goodness of fit index; CFI, comparative fit index; PCFI, parsimony comparative fit index; RMSEA, root mean square error of approximation. Table 3. Alpha coefficients of the subscales. Subscale EDS EDE PCS GPS PFS Items (n) 6 3 4 5 2 Alpha 0.88 0.76 0.85 0.83 0.59 EDS, emergency department staff; EDE, emergency department environment; PCS, physician care satisfaction; GPS, general patient satisfac- tion; PFS, patient family’s satisfaction. Non co mmerc ial us e o nly [Emergency Care Journal 2015; 11:5276] [page 45] faction with nurses or quality of care). A recent meta-analysis20 suggested that the rate of patient satisfaction in hospital emer- gency rooms ranged between 24 and 98.4%. While the meta-analytic review reported desir- able levels of satisfaction with hospital emer- gency rooms’ performance on a national scale, a large variance was detected in the reported rates. As a result, there is a growing need for quality improvement and betterment of ED performance. Since the psychometrically cor- rect assessment of satisfaction is imperative for quality improvement purposes, BEPSS may be nationally utilized in order for assessment of ED patient satisfaction. Several limitations of the present study are worth noting. First, the findings of this study are limited to the cultural context of the study. Cross-cultural differences may play a remark- able role in perception of healthcare quality from patients’ points of view. Therefore, cross- cultural research is encouraged to investigate the factor structure and psychometric proper- ties of this newly developed scale. Second, only about 37% of the participants were females. Using probability-sampling methods such as stratified random sampling could potentially regulate the gender distribution according to the target population. Third, triage category was not directly correlated with satisfaction. Conclusions In sum, this study provided evidence for the 5-factor structure of ED patient satisfaction as measured by BEPSS. All five subscales, as developed in the original paper, are appropri- ate for use in EDs of hospitals in order to measure patient satisfaction in five domains. Future research is recommended to utilize this scale in different hospitals and across different languages and cultures as the current findings are limited to the patients with addressed characteristics. References 1. Andaleeb SS. Service quality perceptions and patient satisfaction: a study of hospi- tals in developing country. Soc Sci Med 2001;52:1359-70. 2. Rahmqvist M, Bara AC. Patient character- istics and quality dimensions related to patient satisfaction. Int J Qual Health C 2010;22:86-92. 3. DiMatteo MR, DiNicola DD. Achieving patient compliance: the psychology of the medical practitioner’s role. New York, NY, USA: Pergamon Press; 1982. 4. Ley P. Satisfaction, compliance and com- munication. Brit J Clin Psychol 1982;21: 241-54. 5. Rubin H, Wu A. Patient satisfaction: its importance and how to measure it. In: Gitnick G, ed. The business of medicine: a physician’s guide. New York, NY, USA: Elsevier; 1991. pp 397-409. 6. Atari M, Akbari-Zardkhaneh S, Atari M, Naderi-far N. Development and validation of the preliminary version of brief inpa- tient satisfaction scale (BISS). Int J Hosp Res 2014;3:263-8. 7. Boudreaux ED, O’Hea EL. Patient satisfac- tion in the emergency department: a review of the literature and implications for practice. J Emerg Med 2004;26:13-26 8. Abbasi S, Farsi D, Bahrani M, et al. Emergency medicine specialty may improve patient satisfaction. Med J Islam Repub Iran 2014;28:61-70. 9. Omidvari S, Shahidzadeh A, Montazeri A, et al. Patient satisfaction survey in the hospitals of Tehran University of medical Sciences, Tehran, Iran. Paiesh, Health Sciences Journal of Jehad Daneshgahi 2008;2:141-52. 10. Soleimanpour H, Gholipouri C, Salarilak S, et al. Emergency department patient satis- faction survey in Imam Reza Hospital, Tabriz, Iran. Int J Emerg Med 2011;4:1-7. 11. Mortazavi A, Kazemi M, Shirazi A, et al. The relationships between patient satis- faction and loyalty in the private hospital industry. Iran J Public Health 2009;38:60-9. 12. Hashemi B, Baratloo A, Rahmati F, et al. Emergency department performance indexes before and after establishment of emergency medicine. Emergency 2013; 1:20-3. 13. Atari M, Bahrami-Ehsan H, Atari M. Developing a psychometric scale for brief evaluation of outpatient satisfaction. Int J Hosp Res 2015;2:55-62. 14. Nervetti G, Milanesi A, Motta M, et al. Patients suffering benign chronic pain becoming acute: ER approach. Emerg Care J 2006;2:22-7. 15. Atari M, Atari M. Brief emergency depart- Article Figure 1. Confirmatory factor analysis of the brief emergency department patient’s satis- faction scale. Non co mmerc ial us e o nly [page 46] [Emergency Care Journal 2015; 11:5276] ment patient satisfaction scale (BEPSS); development of a new practical instru- ment. Emergency 2015;3:103-8. 16. Byrne BM. Structural equation modeling with EQS: basic concepts, applications and programming 2nd ed. Mahwah, NJ, USA: Erlbaum; 2006. 17. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alterna- tives. Struct Equ Modeling 1999;6:1-55. 18. Taylor C, Benger JR. Patient satisfaction in emergency medicine. Emerg Med J 2004;21:528-32. 19. Rahmati F, Gholamalipoor H, Hashemi B, et al. The reasons of emergency depart- ment patients’ dissatisfaction. Iran J Emerg Med 2015;2:1-5. 20. Kardanmoghadam V, Movahednia N, Movahednia M, et al. Determining patients’ satisfaction level with hospital emergency rooms in Iran: a meta-analy- sis. Glob J Health Sci 2015;7:260-9. Article Non co mmerc ial us e o nly