Archives of Academic Emergency Medicine. 2021; 9(1): e47 OR I G I N A L RE S E A RC H Emergency Department Quality Dashboard; a Systematic Review of Performance Indicators, Functionalities, and Challenges Sohrab Almasi1, Reza Rabiei1, Hamid Moghaddasi1∗, Mojtaba Vahidi-Asl2 1. Department of Health Information Technology and Management, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran. 2. Faculty of Computer Science and Engineering, Shahid Beheshti University, Tehran, Iran. Received: April 2021; Accepted: May 2021; Published online: 17 June 2021 Abstract: Introduction: Effective information management in the emergency department (ED) can improve the control and management of ED processes. Dashboards, known as data management tools, efficiently provide informa- tion and contribute greatly to control and management of ED. This study aimed to identify performance indica- tors quality dashboard functionalities, and analyze the challenges associated with dashboard implementation in the ED. Methods: This systematic review began with a search in four databases (Web of Science, PubMed, Embase, and Scopus) from 2000 to May 30, 2020, when the final search for papers was conducted. The data were collected using a data extraction form and the contents of the extracted papers were analyzed through ED performance indicators, dashboard functionalities, and implementation challenges. Results: Performance indicators reported in the reviewed papers were classified as the quality of care, patient flow, timeliness, costs, and resources. The main dashboard functionalities noted in the papers included reporting, customization, alert creation, resource management, and real-time information display. The dashboard implementation challenges included data sources, data quality, integration with other systems, adaptability of dashboard functionalities to user needs, and selection of appropriate performance indicators. Conclusion: Quality dashboards facilitate processes, communication, and situation awareness in the ED; hence, they can improve care provision in this department. To enhance the effectiveness and efficiency of ED dashboards, officials should set performance indicators and consider the conformity of dashboard functionalities with user needs. They should also integrate dashboards with other relevant systems at the departmental and hospital levels. Keywords: Emergency service, hospital; quality indicators, health care; data management; systematic review Cite this article as: Almasi S, Rabiei R, Moghaddasi H, Vahidi-Asl M. Emergency Department Quality Dashboard; a Systematic Review of Performance Indicators, Functionalities, and Challenges. Arch Acad Emerg Med. 2021; 9(1): e47. https://doi.org/10.22037/aaem.v9i1.1230. 1. Introduction Since the emergency department (ED) provides complex ser- vices and hosts patients in critical physical conditions, timely and precise service provision in ED is considered a chal- lenge(1). Moreover, the increased number of visits to the ED disrupts emergency care provision in this department(2, 3). These problems lead to negative outcomes for both pa- tients (morbidity and mortality) and healthcare providers in ∗Corresponding Author: Hamid Moghaddasi; Department of Health Infor- mation Technology and Management, School of Allied Medical Sciences, Dar- band St, Qods Square, Shariati St, Tehran, Iran. Phone number: (+98) 21 22747373, Fax: (+98) 21 2272 115, E-mail: Moghaddasi@sbmu.ac.ir, ORCID: https://orcid.org/0000-0002-5906-0329. the ED (stress and burnout)(4). Effective information man- agement in the ED provides timely information and improves process control and management in this department (5). The use of information technology in the ED plays a pivotal role in information management and enhances managerial and treatment-related processes(6). However, such systems are either not used at all or designed in a way that they further increase the workload of healthcare providers in the ED(7). Research shows that although these systems promote coor- dination and communication among healthcare providers, they fail to offer rapid access to patient information, delay information recording, and have poor user-friendliness.(8) A poor user interface design is another disadvantage of ED in- formation systems, which can cause users to face problems in accessing the necessary information (9, 10). Dashboards, This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem S. Almasi et al. 2 known as data management tools in the ED, collect data from various information systems including the ED, laboratory, and radiology information systems and display them based on pre-defined key performance indicators. Using dash- boards, ED managers evaluate the performance of their de- partment, identify problems, analyze their causes, and thus promote their own performance(11). By displaying informa- tion through visual tools, dashboards help managers visually identify trends and patterns. The use of quality dashboards in the ED also facilitates work processes, reduces hospitaliza- tion duration, better demonstrates the ED status, improves coordination, enables rapid access to information, decreases complications for patients, promotes the monitoring of per- formance indicators by managers, enhances reporting flexi- bility, and provides timely information (12, 13). The key to a suitable design is paying attention to functionalities and per- formance indicators monitored by dashboards. Dashboard functionalities demonstrate the system operations or the ac- tivities performed/facilitated by these operations (12). Clas- sified as a type of dashboard, the quality dashboard collects information from various sources and, by using visual tools based on key performance indicators, provides information at the level of department or organization to help users with decision-making (12-14). Quality dashboards include those of the operating room (15), radiology (16), nursing (17), and intensive care unit (18). The use of dashboards enhances management of department processes, improves communi- cations, and thus, aids decision-making. It is important to employ quality dashboards to obtain timely information for effective control and management of ED pro- cesses. For effective information management and commu- nications in the ED, key performance indicators and func- tionalities of the quality dashboards should also be deter- mined in terms of user needs (12, 19). This study aimed to identify ED performance indicators and quality dashboard functionalities, and analyze the challenges associated with their implementation in the ED. 2. Methods 2.1. Data sources and search strategy Data search and extraction phases were performed based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist (20). The search formula was adopted through a combination of MeSH terms, Emtree, and keywords pertaining to dashboards and the ED (Table 1). The search for finding relevant papers was conducted in four databases (Embase, Web of Science, PubMed, and Scopus) (Table 2). The final search for papers was conducted on May 30, 2020. Moreover, this study spanned from June 1 to December 30. One researcher (SA) independently searched and retrieved the papers, whereas uncertainties were discussed with the other two authors (RR and HM). The relevant papers were also retrieved through a search in Google Scholar and Google. The search for papers was finalized with a bibliographic check of the designated papers. 2.2. Inclusion and exclusion criteria The inclusion criteria were: 1) papers written in English; 2) papers on ED quality dashboards; and 3) papers on per- formance indicators and functionalities of ED quality dash- boards. Moreover, the exclusion criteria were: 1) non-English papers; 2) papers merely designing a clinical dashboard for the ED; or 3) papers examining the dashboards of other hos- pital wards. 2.3. Paper selection, paper evaluation, and data extraction In the screening step, three authors (SA, RR, and HM) checked the titles and abstracts of the papers and irrele- vant papers were removed. In the eligibility step, the papers were independently checked by the same noted authors. The bibliography check was then conducted by one of the au- thors (SA). The quality assessment of the papers based on Cochrane Effective Practice and Organization of Care (EPOC) guideline (21) was independently conducted by SA, RR, and HM. For data extraction, the performance indicators of the ED were first extracted from the papers. The performance indi- cators used by the ED quality dashboards mentioned in the reviewed papers were classified as patient flow, timeliness, quality of care, costs, and resources (Table 3) (22). The first author’s name, the year of publication, the place of study, the quality dashboard functionalities, and the main challenges associated with using the dashboard in the ED were then ex- tracted from each paper (Table 4). 3. Results A total of 1275 papers were retrieved through the search in databases. Additionally, four papers were retrieved from a search attempt on Google Scholar and Google. After remov- ing the duplicates using endnote software, 484 papers re- mained. The titles and abstracts of the papers were then reviewed. As a result, 423 papers were removed and 61 re- mained. In the next step, the full texts of papers were checked and this resulted in the removal of 42 papers. Ultimately, 18 papers remained for analysis. One more paper was also re- trieved when performing a bibliographic check of the des- ignated papers. No paper was removed after quality assess- ment, and all the papers entered the final analysis phase. Fi- nally, 19 papers were reviewed in this study. Figure 1 displays the paper selection process. This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem 3 Archives of Academic Emergency Medicine. 2021; 9(1): e47 3.1. Quality assessment According to the quality assessment of papers, three studies (19, 23, 33) were considered as “high quality”; four studies (24, 25, 31, 37) were introduced as “fair to good quality”, and 12 studies (11, 26-30, 32, 34-36, 38, 39) were regarded as low quality (Table 5). 3.2. ED quality dashboard performance indica- tors The performance indicators (26 in total) were reported in five main categories (quality of care, patient flow, timeliness, costs, and resources). The majority of performance indica- tors were related to patient flow (8 indicators), timeliness (7 indicators), quality of care (4 indicators), and costs (2 indica- tors), respectively. The following indicators belonged to the patient flow category: the number of patients discharged and the type of discharge (referral to other centers, admission at the hospital) (in 11 papers), the total number of people vis- iting the ED (n=10), the number of patients in the ED (di- vided by age, gender, type of specialty, and triage level) (n=8), the number of patients admitted per triage level (n=6), how the patients visited the ED (personal vehicles, ambulance, on foot), and the number of patients for whom a decision was made in six hours (n=4). The timeliness category consisted of the following performance indicators: the patients’ mean length of stay (n=16), the time of triaging (per each triage level) (n=4), and the mean time elapsed since the patients’ arrival in the ED until the onset of triaging (n=4). Only three out of the 19 papers had mentioned the quality of care per- formance indicators, which included patient revisits (before 72 hours) (n=3) and the percentage of mortality among the patients admitted to the ED (in=2). The cost performance in- dicators included the number of tests ordered by the doctors (n=8) and the number of consultations given (n=3). Finally, resource performance indicators were the number of beds (available, extra, occupied, reserved, and out-of-order) (n=7) and the number of personnel in the ED (divided by discipline and gender) (n=6). 3.3. ED quality dashboard functionalities All the 19 papers discussed the reporting functionality. This was followed by customization (n=12) (11, 19, 23-32), alerting (n=10) (11, 25, 27, 29-35), resource management (n=9) (19, 24-29, 33, 36), and real-time information display (n=8) (11, 19, 24, 31, 33, 37-39) as the most frequently mentioned func- tionalities, respectively. Other papers also reported function- alities such as automated data collection (n=4) (11, 29, 31, 39) and the use of drill-up, drill-down, and drill-through (n=4) (23, 25, 31, 33). 3.4. Challenges to use of quality dashboards in ED The challenges associated with the use of quality dashboards in the ED included a lack of integration with other hospi- tal systems and inputting the data manually (24, 29), lack of adaptability to work processes in the ED and lack of flexibil- ity (26, 28, 30, 32-34, 37), breaching patient confidentiality by displaying patients’ names and test results on the dash- board’s large screen (27, 31), a problem with understanding and interpreting the type of information displayed through visualization tools (27, 28), and the accuracy of the data en- tered into the dashboard (23). 4. Discussion This study aimed to identify the performance indicators, functionalities, and challenges of quality dashboards in the ED. Some studies had examined the effects of clinical/quality dashboards on patient care improvement or analyzed the functionalities and positive and negative effects of using dashboards in hospital(40, 41). Determining key perfor- mance indicators lays the ground for performance mea- surement, ensures progress evaluation based on pre-defined goals/criteria, and aids managers in decision-making by pro- viding timely and appropriate information (42). Based on the results of this study, ED performance indicators are di- vided into five groups of quality of care, patient flow, time- liness, costs, and resources. These findings are in line with the results of studies by Sørup (22) and the US Institute of Medicine (43). Since the patient-centeredness indicator deals with the provision of care based on patients’ needs, preferences, and values, this indicator was called patient flow in this study (43). Safety deals with the perceived damage and complications of treatment processes. Since the pa- tient mortality indicator may not be related to treatment pro- cesses at the hospital and result from underlying diseases (44, 45), this indicator was replaced by the quality of care in this study. As for the quality dashboard functionalities, our findings yielded reporting, customization, real-time in- formation display, resource management, and alerts. Previ- ous studies have introduced measurement, monitoring, col- lection, processing, performance measurement, and report- ing as dashboard functionalities(40, 42, 46, 47). The ED has complex processes with its personnel having different infor- mation needs with respect to their responsibilities. There- fore, it is essential to pay attention to quality dashboard func- tionalities and select suitable performance indicators to be monitored by the dashboard (40). The dashboard develop- ment challenges included data sources and data quality, in- tegration with other systems, adaptability of dashboard func- tionalities to user needs, and selection of appropriate perfor- mance indicators. These findings are consistent with those This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem S. Almasi et al. 4 of Ghazisaeidi et al. (48). The main challenge in integrating dashboards with other systems is mostly related to the infor- mation technology infrastructure in the organization, which focuses on data collection through different data sources, data integration, and their linkage to the dashboard through the most appropriate method. (49) Designing a suitable architecture for supporting the dashboard requires identi- fication of different data hosting structures, various meth- ods of data replication and transfer, and the best query lan- guage for this data structure (50). Moreover, the identifi- cation of data sources—the processes used for data genera- tion— and precise, comprehensive, and reliable datasets for generating high-quality data are major topics in dashboard development, which increase ED users’ trust in dashboard information (48). Research findings also demonstrate that continuous implementation with small changes can greatly contribute to the success of dashboard in meeting users’ needs(48, 50). 5. Limitations There are limitations with the current study that could be ad- dressed in future research. First of all, the current study re- viewed studies published in English and there might be use- ful relevant studies that were excluded. In addition, although we systematically studied the key performance indicators of quality dashboard in the emergency department, we did not include studies that addressed clinical dashboards of this de- partment due to aim of the study. In the analysis step, an at- tempt was made to avoid or minimize the risk of bias, as any possible discrepancy experienced by either of the first three authors were discussed in group. However, there may have been issues that did not catch our attention. 6. Conclusion and suggestion The findings of the current study indicated that applying key performance indicators for ED quality dashboard, includ- ing quality of care, patient flow, timeliness, costs, and re- sources, could improve utilization management and qual- ity of care. Moreover, a quality dashboard should support functions such as customization, reporting, real-time infor- mation display, resource management, alert creation, and automated data collection. In addition, the use of drill-up, drill-down, and drill-through features could help improve ef- ficiency and effectiveness of ED quality dashboard. With re- spect to the challenges noted, it is suggested that in further developments of ED dashboard, attention should be paid to data sources, the quality of data, and integration with other systems both in the ED and in other hospital departments. 7. Declarations 7.1. Acknowledgments None. 7.2. Funding and Support This systematic review did not receive any funding. 7.3. Author contribution Concept and study design: SA, RR, HM, MVA; literature search, study selection and data collection: SA; analysis and interpretation: SA, RR, HM; Writing the article: SA; critical re- vision of the article: RR, HM; final approval of the article: SA, RR, HM, MVA. 7.4. Ethical approval This review was part of a larger study, approved by the Ethics Committee of Shahid Beheshti University of Medical Sci- ences, Iran (IR.SBMU.RETECH.REC.1399.565). 7.5. Conflict of interest The authors declare that they have no competing interests. References 1. 1. Steptoe AP, Corel B, Sullivan AF, Camargo CA. Character- izing emergency departments to improve understanding of emergency care systems. Int J Emerg Med. 2011;4(1):42. 2. Higginson I. Emergency department crowding. Emer- gency medicine journal : EMJ. 2012;29(6):437. 3. Morley C, Unwin M, Peterson GM, Stankovich J, Kins- man L. Emergency department crowding: A systematic review of causes, consequences and solutions. PLoS One. 2018;13(8):e0203316. 4. Taylor TB. Information management in the emergency department. Emerg Med Clin N Am. 2004;22(1):241-57. 5. Ahanhanzo YG, Kpozehouen A, Sopoh G, Sossa-Jérôme C, Ouedraogo L, Wilmet-Dramaix M. Management of information within emergencies departments in devel- oping countries: analysis at the National Emergency Department in Benin. Pan Afr Med J. 2016;24:263-. 6. Selck FW, Decker SL. Health Information Technology Adoption in the Emergency Department. Health Serv Res. 2016;51(1):32-47. 7. Callen J, Paoloni R, Li J, Stewart M, Gibson K, Georgiou A, et al. Perceptions of the effect of information and com- munication technology on the quality of care delivered in emergency departments: a cross-site qualitative study. Ann Emerg Med. 2013;61(2):131-44. 8. Handel DA, Wears RL, Nathanson LA, Pines JM. Using information technology to improve the quality and safety of emergency care. Academic emergency medicine : This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem 5 Archives of Academic Emergency Medicine. 2021; 9(1): e47 official journal of the Society for Academic Emergency Medicine. 2011;18(6):e45-51. 9. Fairbanks RJ, Guarrera TK, Karn KS, Caplan SH, Shah MN, Wears RL. Interface Design Characteristics of a Pop- ular Emergency Department Information System. Proc Hum Factors Ergon Soc Annu Meet. 2008;52(12):778-82. 10. Wilbanks BA, Langford PA. A review of dashboards for data analytics in nursing. Computers, informatics, nursing : CIN. 2014;32(11):545-9. 11. Yoo J, Jung KY, Kim T, Lee T, Hwang SY, Yoon H, et al. A real-time autonomous dashboard for the emergency department: 5-year case study. JMIR mHealth uHealth. 2018;6(11):e10666. 12. Pauwels K, Ambler T, Clark BH, LaPointe P, Reibstein D, Skiera B, et al. Dashboards as a Service: Why, What, How, and What Research Is Needed? J Serv Res. 2009;12(2):175- 89. 13. Carroll C, Flucke N, Barton AJ. The use of dashboards to monitor quality of care. Clinical nurse specialist CNS. 2013;27(2):61-2. 14. Randell R, Alvarado N, McVey L, Ruddle RA, Doherty P, Gale C, et al. Requirements for a quality dashboard: Lessons from National Clinical Audits. AMIA Annu Symp Proc. 2020;2019:735-44. 15. Park KW, Smaltz D, McFadden D, Souba W. The oper- ating room dashboard. The Journal of surgical research. 2010;164(2):294-300. 16. Karami M, Safdari R. From Information Manage- ment to Information Visualization. Appl Clin Inform. 2016;7(02):308-29. 17. Russell M, Hogg M, Leach S, Penman M, Friel S. Developing a general ward nursing dashboard. Nursing standard (Royal College of Nursing (Great Britain) : 1987). 2014;29(15):43-9. 18. Koch SH, Weir C, Westenskow D, Gondan M, Agutter J, Haar M, et al. Evaluation of the effect of information inte- gration in displays for ICU nurses on situation awareness and task completion time: A prospective randomized controlled study. Int J Med Inform. 2013;82(8):665-75. 19. Martin N, Bergs J, Eerdekens D, Depaire B, Verelst S. Developing an emergency department crowding dash- board: A design science approach. Int Emerg Nurs. 2018;39:68-76. 20. Moher D, Liberati A, Tetzlaff J, Altman DG. Pre- ferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS medicine. 2009;6(7):e1000097. 21. Cochrane Effective Practice and Organisation of Care (EPOC). Suggested risk of bias criteria for EPOC reviews. EPOC Resources for review au- thors,2017. Cited[20 September 2020]. Available at: http://epoc.cochrane.org/resources/epoc-resources- review-authors. 22. Sørup CM, Jacobsen P, Forberg JL. Evaluation of emer- gency department performance - a systematic review on recommended performance and quality-in-care mea- sures. Scand J Trauma Resusc Emerg Med. 2013;21:62-. 23. van Deen WK, Cho ES, Pustolski K, Wixon D, Lamb S, Valente TW, et al. Involving end-users in the design of an audit and feedback intervention in the emergency department setting–a mixed methods study. BMC health services research. 2019;19(1):270. 24. Patterson ES, Rogers ML, Tomolo AM, Wears RL, Tsevat J. Comparison of extent of use, information accuracy, and functions for manual and electronic patient status boards. Int J Med Inform. 2010;79(12):817-23. 25. Hester G, Lang T, Madsen L, Tambyraja R, Zenker P. Timely data for targeted quality improvement interven- tions: use of a visual analytics dashboard for bronchiolitis. Appl Clin Inform. 2019;10(01):168-74. 26. Franklin A, Gantela S, Shifarraw S, Johnson TR, Robinson DJ, King BR, et al. Dashboard visualizations: Supporting real-time throughput decision-making. Jour- nal of biomedical informatics. 2017;71:211-21. 27. McLeod B, Zaver F, Avery C, Martin DP, Wang D, Jessen K, et al. Matching capacity to demand: a regional dashboard reduces ambulance avoidance and improves accessibility of receiving hospitals. Academic emergency medicine : official journal of the Society for Academic Emergency Medicine. 2010;17(12):1383-9. 28. France DJ, Levin S, Hemphill R, Chen K, Rickard D, Makowski R, et al. Emergency physicians’ behaviors and workload in the presence of an electronic white- board. International journal of medical informatics. 2005;74(10):827-37. 29. Staib A, Sullivan C, Jones M, Griffin B, Bell A, Scott I. The ED-inpatient dashboard: Uniting emergency and inpatient clinicians to improve the efficiency and qual- ity of care for patients requiring emergency admission to hospital. Emergency medicine Australasia : EMA. 2017;29(3):363-6. 30. Hertzum M, Simonsen J. Effects of electronic emergency-department whiteboards on clinicians’ time distribution and mental workload. Health informatics journal. 2016;22(1):3-20. 31. Mazor I, Heart T, Even A. Simulating the impact of an online digital dashboard in emergency departments on patients length of stay. Journal of Decision Systems. 2016;25(sup1):343-53. 32. McGeorge N, Hegde S, Berg RL, Guarrera-Schick TK, LaVergne DT, Casucci SN, et al. Assessment of In- novative Emergency Department Information Displays in a Clinical Simulation Center. J Cogn Eng Decis Mak. 2015;9(4):329-46. This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem S. Almasi et al. 6 33. Dexheimer JW, Kennebeck S. Modifications and integration of the electronic tracking board in a pedi- atric emergency department. Pediatric emergency care. 2013;29(7):852-7. 34. Bisantz A, Pennathur P, Guarrera T, Fairbanks R, Perry S, Zwemer F, et al. Emergency Department Status Boards: A Case Study in Information Systems Transition. J Cogn Eng Decis Mak. 2010;4:39-68. 35. Rasmussen R, Fleron B, Hertzum M, Simonsen J, editors. Balancing tradition and transcendence in the implementation of emergency-department electronic whiteboards. Selected papers of the information systems research seminar in Scandinavia; 2010. 36. Aronsky D, Jones I, Lanaghan K, Slovis CM. Supporting patient care in the emergency department with a com- puterized whiteboard system. J Am Med Inform Assoc. 2008;15(2):184-94. 37. Pennathur PR, Bisantz AM, Fairbanks RJ, Perry SJ, Zwemer F, Wears RL, editors. Assessing the impact of com- puterization on work practice: Information technology in emergency departments. Proceedings of the Human Factors and Ergonomics Society Annual Meeting; 2007: SAGE Publications Sage CA: Los Angeles, CA. 38. Boger E. Electronic tracking board reduces ED pa- tient length of stay at Indiana Hospital. J Emerg Nurs. 2003;29(1):39-43. 39. Wears RL, Perry SJ, Shapiro M, Beach C, Croskerry P, Behara R, editors. A comparison of manual and elec- tronic status boards in the emergency department: what’s gained and what’s lost? Proceedings of the Human Factors and Ergonomics Society Annual Meeting; 2003: SAGE Publications Sage CA: Los Angeles, CA. 40. Randell R, Greenhalgh J, Wyatt J, Gardner P, Pearman A, Honey S, et al. Electronic whiteboards: review of the literature. Studies in health technology and informatics. 2015;210:389-93. 41. Dowding D, Randell R, Gardner P, Fitzpatrick G, Dykes P, Favela J, et al. Dashboards for improving patient care: review of the literature. Int J Med Inform. 2015;84(2):87- 100. 42. Janssen A, Donnelly C, Kay J, Thiem P, Saavedra A, Pathmanathan N, et al. Developing an Intranet-Based Lymphedema Dashboard for Breast Cancer Multidisci- plinary Teams: Design Research Study. J Med Internet Res. 2020;22(4):e13188. 43. Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington (DC): National Academies Press (US); 2001. 44. Diley I, Badrinath P, Annon S. Is mortality a good indicator of the clinical quality of National Health Service hospitals? A cross-sectional study of outlier trusts for mortality indices using quality dashboards. JRSM open. 2014;5(8):2054270414533325. 45. Ngantcha M, Le-Pogam M-A, Calmus S, Grenier C, Evrard I, Lamarche-Vadel A, et al. Hospital quality mea- sures: are process indicators associated with hospital standardized mortality ratios in French acute care hospi- tals? BMC health services research. 2017;17(1):578-. 46. Eckerson WW. Performance dashboards: measuring, monitoring, and managing your business: John Wiley & Sons; 2010. 47. Jeffs L, Beswick S, Lo J, Lai Y, Chhun A, Campbell H. Insights from staff nurses and managers on unit-specific nursing performance dashboards: a qualitative study. BMJ quality & safety. 2014;23(12):1001-6. 48. Ghazisaeidi M, Safdari R, Torabi M, Mirzaee M, Farzi J, Goodini A. Development of performance dashboards in healthcare sector: key practical issues. Acta Informatica Medica. 2015;23(5):317. 49. Minnigh TR, Gallet J. Maintaining quality control using a radiological digital X-ray dashboard. J Digit Imaging. 2009;22(1):84-8. 50. Rasmussen NH, Bansal M, Chen CY. Business dash- boards: a visual catalog for design and deployment: John Wiley & Sons; 2009. This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem 7 Archives of Academic Emergency Medicine. 2021; 9(1): e47 Figure 1: Flow diagram based on PRISMA. Table 1: Search formula No. Terms Search formula 1 Dashboard “dashboard” OR “electronic whiteboard” OR “emergency department dashboard” OR “emergency department information system” OR “status board” OR “electronic tracking board” 2 ED “emergency department” OR “emergency” OR “emergency medicine” OR “emergency medical services” OR “emergency unit” 3 1 and 2 ED: emergency department. This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem S. Almasi et al. 8 Table 2: Search strategy Database Search terms PubMed (Emergency department dashboard[TIAB] OR whiteboard[TIAB] OR emergency department information sys- tem[TIAB] OR status board[TIAB] OR Electronic tracking board[TIAB] OR dashboard[TIAB]) AND (emergency department[MeSH] OR emergency department[TIAB] OR emergency[TIAB] OR emergency medicine[TIAB] OR Emergency Medical Services[MeSH] OR emergency unit[TIAB]) NOT review Web of Science TS=(dashboard OR "electronic whiteboard" OR “Emergency department dashboard” OR “emergency depart- ment information system” OR "status board" OR "Electronic tracking board") AND TS=("emergency depart- ment" OR emergency OR "emergency medicine" OR "Emergency Medical Services" OR "emergency unit") Embase (’emergency department dashboard’:ab,ti OR whiteboard:ab,ti OR ’emergency department information sys- tem’:ab,ti OR dashboard:ab,ti OR ’electronic tracking board’:ab,ti OR ’status board’:ab,ti) AND (’emergency ward’/exp/mj OR ’emergency medicine’/exp/mj OR ’emergency health service’/exp/mj OR ’emergency ward’:ab,ti OR ’emergency medicine’:ab,ti OR ’emergency health service’:ab,ti) Scopus ((TITLE-ABS (dashboard) OR TITLE-ABS ( "electronic whiteboard" ) OR TITLE-ABS ( "Emergency department dashboard" ) OR TITLE-ABS ( "emergency department information system" ) OR TITLE-ABS ( "status board" ) OR TITLE-ABS ( "Electronic tracking board" ) ) AND ( TITLE-ABS ( "emergency department" ) OR TITLE-ABS ( emergency ) OR TITLE-ABS ( "emergency medicine" ) OR TITLE-ABS ( "Emergency Medical Services" ) OR TITLE-ABS ( "emergency unit" ) ) ) Table 3: Evaluated key performance indicators of emergency department in included articles Authors Patient flow Timeliness 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Van Deen et al. (23) p p p p Hester et al.(24) p p p Martin et al. (19) p p p p p p p p Yoo et al. (11) p p Franklin et al. (25) p p p p p p p p Staib et al. (26) p Hertzum et al. (27) p p p p Mazor et al. (28) p p p p p p McGeorge et al. (29) p p p p p p p p Dexheimer et al. (30) p p p p p Mcleod et al. (31) p p p p Bisantz et al. (32) p p p p p p p p Patterson et al. (33) p p p p p Rasmussen et al. (34) p p Aronsky et al. (35) p p p Pennathur et al. (36) p p France et al. (37) p p p Wears et al. (38) p p p Boger (39) p p p p p 1. The total number of people visiting the ED 2. How patients visit the ED 3. The number of patients discharged, and the type of discharge 4. The number of patients based on the triage level 5. The number of patients leaving the ED without primary evaluation and treatment 6. The number of patients in the ED 7. The number of patients waiting to be visited 8. The number of patients discharged from the ED in six hours 9. The patients’ mean length of stay 10. The mean time elapsed since the doctor’s request for admission until the assignment of a room/bed to the patient in the inpatient ward 11 . The time of triaging (per each triage level) 12. The mean time elapsed since the patients’ arrival in the ED until the onset/beginning of triaging 13. The mean time elapsed since the patients’ arrival in the ED until admission at the ward 14. The mean time elapsed since the patients’ arrival in the ED until doctor’s visit 15. The mean time elapsed since the orders are recorded until the results (tests, imaging, electrocardiography) are ready. This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem 9 Archives of Academic Emergency Medicine. 2021; 9(1): e47 Table 3: Evaluated key performance indicators of emergency department in included articles (continued) Authors Quality of Care Costs Resources 1 2 3 4 5 6 7 8 9 10 Van Deen et al. (23) p p p Hester et al.(25) p p p Martin et al. (19) p p Yoo et al. (11) p p p Franklin et al. (26) ! p p Staib et al. (29) p p p p Hertzum et al. (30) p Mazor et al. (31) p McGeorge et al. (32) p p p p Dexheimer et al. (33) Mcleod et al. (27) Bisantz et al. (34) p p Patterson et al. (24) p p Rasmussen et al. (35) p p Aronsky et al. (36) p p p p Pennathur et al. (37) p France et al. (28) Wears et al. (39) p p Boger (38) 1. Patients readmission (before 72 hours) 2. The percentage of mortality among the patients admitted to the ED 3. The number of heart attacks in 24 hours among the patients admitted through the ED 4. The percentage of mortality among the hospital’s patients admitted through the ED 5. The number of visits made 6. The mean number of visits made by the doctor per hour 7. The number of tests ordered by the doctors 8. The number of consultations given 9. The number of personnel in the ED (based on discipline and sex) 10. The number of beds (available, extra, occupied, reserved, out-of-order) This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem S. Almasi et al. 10 Table 4: Specifications of the dashboards examined in the papers Author Quality dashboard functionali- ties Challenges Van Deen et al. (23) 2019; USA Reporting Drill-down and drill-up Drill-through Customization The data entered into the dashboard were not reliable or sufficiently ac- curate. Hester et al.(24) 2019;Denmark Reporting Real-time information display The designed dashboard functionalities were not compatible with the work processes of the healthcare providers in the ED. Martin et al. (19) 2018; USA Resource management Reporting Real-time information display Customization N/A Yoo et al. (11) 2018; South Korea Real-time information display Au- tomated data collection Reporting Alert Customization The designed dashboard was evaluated only in one ED, and it might not be useful because of the hospital’s different specialties and the use of dif- ferent performance indicators. Franklin et al. (25) 2017; USA Resource management Alert Re- porting Real-time information display Drill-down and drill-up Drill-through The information displayed by the dashboard did not meet the diverse needs of ED users. Staib et al. (26) 2017; Aus- tralia Real-time information display Re- porting N/A Hertzum et al. (27) 2016; Denmark Resource management Reporting Real-time information display Customization The challenges included a lack of integration between the dashboard and the other systems at the hospital, and inputting the information manually. Mazor et al. (28) 2016; Is- rael Alert Resource management Re- porting Customization Drill-down and drill-up Drill-through7 The research limitation was that the evaluation was performed in a simu- lated environment. McGeorge et al. (29) 2015; USA Reporting Resource management Customization The research limitation was that the evaluation was performed in a simu- lated environment. Dexheimer et al. (30) 2013; USA Resource management Alert Re- porting Customization A problem in interpreting the information displayed by the dashboard, and breaching confidentiality as some patient demographic information was displayed in the dashboard. Mcleod et al. (31) 2010; Canada Real-time information display Au- tomated data collection Reporting N/A Bisantz et al. (32) 2010; USA Resource management Reporting Customization A problem with understanding the type of information displayed in the dashboard, and the dashboard’s lack of adaptability to work processes and the doctors’ information needs Patterson et al. (33) 2010; USA Automated data collection Re- source management Alert Report- ing Customization Lack of integration with other hospital systems, and inputting the data manually Rasmussen et al. (34) 2010;Denmark Alert Reporting Customization Lack of compatibility between the features of the new system and the work processes in the ED Aronsky et al. (35) 2007; USA Real-time information display Au- tomated data collection Alert Re- porting Customization Drill-down and drill-up Drill-through Displaying information such as patient name and test results on the dash- board large screen violated patient information confidentiality. Pennathur et al. (36) 2007; USA Alert Reporting Customization Lack of compatibility with workflows, especially during triaging and tracking the stages of care France et al. (37) 2005; USA Resource management Reporting N/A Wears et al. (38) 2003; USA Alert Reporting Low flexibility, no customization feature Boger (39) 2003; USA Reporting Alert N/A ED: emergency department. This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem 11 Archives of Academic Emergency Medicine. 2021; 9(1): e47 Table 5: Risk of bias assessment in included studies based on effective practice and organization of care (EPOC) tools Variable 1 2 3 4 5 6 7 Van Deen et al. (23) p p p p p p p Hester et al.(24) p * p p * * p Martin et al. (19) p p p p p p p Yoo et al. (11) * * * * * p p Franklin et al. (25) * p p p * * p Staib et al. (26) * * p * * p p Hertzum et al. (27) p * p * p * p Mazor et al. (28) * * p * p * p McGeorge et al. (29) * p * * p * p Dexheimer et al. (30) * p * p * * p Mcleod et al. (31) p * p p * p p Bisantz et al. (32) p * * p * * p Patterson et al. (33) p p p p p p p Rasmussen et al. (34) * p * * * p p Aronsky et al. (35) p p p * p * p Pennathur et al. (36) p * p * p * p France et al. (37) * p p * p * p Wears et al. (38) * p * p p * p Boger (39) p p p * * p * 1. Intervention independent of other changes 2. Shape of the intervention effect pre-specified 3. Intervention unlikely to affect data collection 4. Knowledge of the allocated interventions adequately prevented during the study 5. Incomplete outcome data adequately 6. Selective outcome reporting 7. Other risks of bias This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem Introduction Methods Results Discussion Limitations Conclusion and suggestion Declarations References