Hrev_master Abstract The Audit and Feedback process (A&F) is commonly accepted as a good way to improve quality in health care, also in Emergency Departments (ED), where health aspects and pathologies are very different, usually acute and highly complex. Within an Italian Ministry of Health research project called EASY-NET, we con- ducted a systematic review of literature on A&F in EDs from 2014 to December 2019 to evaluate the impact of this approach in a par- ticular setting where time-dependent indicators are fundamental. We selected 24 articles: 9 about infective pathologies (i.e. antibiot- ic stewardship), 6 about cardiovascular acute emergencies (i.e. car- diac arrest), 2 about stroke, 3 about laboratory tests, and 4 about other fields (i.e. diabetic ketoacidosis or use of prothrombin com- plex). Most of articles proposed a multimodal approach: only 7 concerned A&F alone. Despite the wide range on interventions modality and the poor comparability of the considered studies, the results are encouraging and confirm the importance to implement A&F both in emergency and in other clinical settings. Background The use of Audit and Feedback (A&F) is commonly accepted as a tool to improve quality in all fields of medicine.1 It can also be used to evaluate changes in care provision or to confirm that cur- rent practice meets the expected level of performance.2 A review made by Ivers et al. in 2012, about all medical fields, showed how the efficacy of intervention was higher when the base- line performance was low, the feedback was made by a supervisor or a colleague and was delivered in both oral and written formats with explicit indications on action plans and targets.3 In emergency care, audit can concern both medical practice (following guidelines, drug prescriptions etc.), or patient point of view (satisfaction, waiting time, etc.). Even if many studies were published about that in the last 20 years, there is still lack of information and a not unique agreement about the importance and validity of A&F in Emergency Departments (EDs). This maybe be due to the peculiarity of EDs environment, which involves many health aspects and pathologies, most of the time in acute high complexity presentation. A review conducted by Rogers et al. (2014) based on A&F in EDs concluded with positive results for interventions on different clinical conditions, but with not enough data for a standardized meta-analysis.4 This review included studies published till January 2014. With regard to A&F, in 2019 Italian Ministry of Health approved a research project called Easy-Net (NET-2016- 02364191), a network composed by 7 teams of seven different research centers each one dislocated in a different Italian Region (Lazio, Friuli Venezia Giulia, Piemonte, Emilia Romagna, Lombardia, Calabria). The aim of the project is to spread the knowledge of A&F in different fields of Italian health system and to promote A&F application thus increasing quality and access of health assistance. Each research center has a research sub-project based on specific clinical areas: cancer, emergency, gynecology, rehabilitation, etc. Our center, dislocated in Friuli Venezia Giulia, has a sub-project called Audit and feedback: efficacy for enhanc- ing the clinical practice and to reduce avoidable differences in the field of emergency medicine, with particular regard to heart attack, stroke and trauma. Major details about the EASY-NET project can be found at https://easy-net.info/ Based on our sub-project, we performed a review of the litera- ture on A&F in EDs from 2014 to December 2019. Our main out- come was to identify how A&F process can influence medical and, in general, health care people behavior in emergency. Methods This work was performed in adherence with the Preferred Reporting Items for Systematic Reviews and Meta-analyses state- ment.5 We performed a systematic review of literature starting from January 2014 to December 2019 on the major medical data- Emergency Care Journal 2020; volume 16:9201 Correspondence: Sabrina Licata, Istituto di Igiene ed Epidemiologia Clinica, Azienda Sanitaria Universitaria Friuli Centrale, Via Colugna 50, 33100, Udine, Italy. Tel.: +39 3397392782 E-mail: sabrina.licata@uniud.it Key Words: Audit and Feedback; Emergency Departments; Time- Dependent Indicators; Interrupted Time Series. Funding: This work was funded by the Italian Ministry of Health (Ministero della Salute) and co-funded by the participating Regions Lazio, Friuli Venezia Giulia, Piemonte, Emilia-Romagna, Lombardia, Calabria through the Programma di rete Ricerca Finalizzata, Bando 2016 - NET-2016-02364191. Conflict of interest: The author declares no potential conflict of interest. Ethics approval and consent to participate: not applicable. Received for publication: 21 June 2020. Accepted for publication: 15 September 2020. This work is licensed under a Creative Commons Attribution 4.0 License (by-nc 4.0). ©Copyright: the Author(s), 2020 Licensee PAGEPress, Italy Emergency Care Journal 2020; 16:9201 doi:10.4081/ecj.2020.9201 [Emergency Care Journal 2020; 16:9201] [page 137] Audit and Feedback in emergency: A systematic review and an Italian project to investigate and improve quality of care Sabrina Licata, Annarita Tullio, Francesca Valent Istituto di Igiene ed Epidemiologia Clinica, Azienda Sanitaria Universitaria Integrata di Udine, Udine, Italy Non -co mmerc ial us e o nly base: PubMed, Cochrane library, CINAHL, OVID clinical Edge. The research included all articles about A&F in EDs. The searched terms are listed in Table 1. Each screening passage has been made by 2 contemporarily reviewers, as described in Figure 1. When there was disagreement, the third author’s opinion was taken into account for final decision. We found 553 articles, identified through search on databases: 321 were duplicates or not pertinent, so we screened 232 abstracts, of whom 64 were chosen and relative full text were searched. Six full text consisted in reviews and we decided to exclude them from our analysis and search for full texts of cited studies that were published between 2014 and 2019, if pertinent. Screening of full texts led us to 24 articles that became the object of our analysis (indicated as “selected studies”). Studies were only included if they met the following eligibili- ty/ inclusion criteria: i) the study used both A&F, ii) the study was conducted in EDs, both exclusively or as a multidisciplinary inter- vention, iii) the study has clear results, both qualitative or quanti- tative, iv) the study was prospective. All articles were published in English language. We also reviewed the reference lists of selected articles to identify addition- al studies for inclusion. Due to the heterogeneity and sometimes to the lack of stan- dardized methods used to perform statistical analysis in selected studies, we conducted a descriptive analysis of the reported results and a qualitative comparison of the different reported A&F approaches. Results We chose 24 articles,6-29 most of whom have been conducted in USA. All 24 studies followed the before/after scheme and 5 have a Plan-Do-Study-Act (PDSA) design.8-10,22 While 9 studies involved different medical departments,6,10-12,17,19,21,22,27 15 of them concerned only Emergency departments and/or paramedics. Most of articles proposed a multimodal approach: only 7 of them concerned exclu- sively A&F.7-11,13,22 As it’s shown in Table 2, 9 articles talked about measurements for infective pathologies (especially antibiotic stew- ardship),7,8,12,13,16,17,21,22,29 6 of them treated cardiac arrest or cardio- vascular emergency problems,8,19,20,25-27 2 of them treated stroke,15,24 3 ones were about laboratory analyses10,18,28 and 4 of them treated any other fields (diabetic ketoacidosis, use of a prothrombin com- plex, etc.), (Table 2). Based on A&F approach, most of articles report a multimodal A&F method. In 15 studies, there were mul- tiple reports (basically weekly or monthly reports); meetings and frontal lessons were used in 18 studies; surveys were used in 6 studies; 9 studies used printed materials (leaflets, posters, hold in pocket charts); in 9 studies, there were other electronic methods (Table 3). All studies declared an improvement due to the interven- tions. Discussion A&F is a common accepted method to assess quality in health- care organizations,1-4 alone or in combination with other interven- tions, to improve health professionals’ performance and encourage the health care people in following professional standards. The auditing consists in a measurement of an individual’s (or specific group’s) professional practice or performance and then in a comparison of the results to professional standards or targets. The results of this comparison are then sent to the individual or the group as a feedback.3,30,31 Even if there have been different attempts to standardize A&F management and processing,32-34 there is still a lot of heterogeneity in literature concerning the correct methodology to collect and analyze data and for the choice of right indicators. The Italian Friuli Venezia Giulia region (FVG) takes part to a national project to evaluate the A&F approach applied to different clinical areas. The FVG sub-project is related to emergency departments with a particular regard to big emergency situations as hearth attack (MI), stroke and major traumas. We conducted a review on main medical electronic databases to understand better the state of art on this topic. Our main outcome was to identify how A&F process can influence medical and, in general, health care people behavior in the field of emergency. In general, we observed a great variability between the articles concerning pathologies or clinical aspects to Review Figure 1. Flow chart of different research phases. Table 1. terms used to perform research. Audit Feedback Audit and feedback Audit emergency feedback emergency Audit and feedback emergency Audit emergency department Feedback emergency department Audit and feedback emergency department Audit “emergency department” Feedback “emergency department” Audit and feedback “emergency department” Audit emergency care Feedback emergency care Audit and feedback emergency care Audit “emergency care” Feedback “emergency care” Audit and feedback “emergency care” Audit emergency unit Feedback emergency unit Audit and feedback emergency unit Audit “emergency unit” Feedback “emergency unit” Audit and feedback “emergency unit” [page 138] [Emergency Care Journal 2020; 16:9201] Non -co mmerc ial us e o nly [Emergency Care Journal 2020; 16:9201] [page 139] whom they were direct, involved departments, clinical figures and practitioners, countries in which studies were conducted and levels of assistance, type and methods of A&F with different kind of audit, indicators, feedback, time frames, statistical analyses, abun- dance of involved figures. This poor comparability of the studies and, in some cases, little information on every specific phase lead us to the impossibility to define a clear conclusion. Anyway, despite the wide range on interventions and the dif- ferences listed above, the results of the A&F approach in emer- gency generally show an improvement of the features considered in selected studies, and a statistically significant improvement for the majority of indicators. The observed improvement refers to: i) clinical outcomes, i.e. the reduction of duration of diabetic ketoacidosis (p<0.001),11 the improvement of survival of patients undergoing a cardiac arrest with a shockable initial rhythm after MI (p=0.02),27 improvement of neurological recovery of patients survived to a cardiac arrest compared to the historical controls (81% vs. 50%, p<0.05);27 ii) appropriateness of clinical test and procedure, i.e. decrease in num- ber of duplicate blood chemistry tests per 100 ED visits (p<0.0001), reduction in both the daily number of laboratory stud- ies (−36.3%; p<0.05),18 (p<0.0001)28 and in POC tests (p<0.0001);28 iii) protocols compliance, i.e. reported by Bentley et al.8 with an 74.3% improvement of sepsis detection and manage- ment, by Hecker et al.13 with an increase from 44 to 68% to adher- ence to cystitis and pyelonephritis management guidelines, and to confidence by the operators. With regard to this last point, Scott et al.29 reported that 100% of the participants were more likely to identify the indication for Indwelling Urethral Catheterization (IUC) insertion and to use incontinence pads (72%), commodes (84%), straight catheterization (78%), urinals/bedpans (89%), and restrooms (67%) as alternatives to IUCs after the intervention; 84% changed the way they approached bladder management in their practice after the intervention; 83% were more likely to rethink the use of IUCs among patients with altered mental status, inpatient boarders, and nursing home patients.29 Taylor et al.17 reported a high perception of value and satisfac- tion from participants even if attendance and participation were frequently limited by time barriers and competing clinical duties. In an A&F context is essential, as always in clinical settings, the full support of colleagues and a close collaboration to ensure that strategies could be successfully implemented in the complex ED setting,16 as it was already highlighted from Ivers et al.3 and Rogers et al.4 Another point of discussion is the fact that A&F intervention is often carried out after the introduction of something new: that can make more difficult to completely understand the impact of the A&F intervention compared to the innovation. Probably both of these novelties work together. As described by Pellis et al.,27 the introduction of Standard Operative Procedures (SOPs) for Target Temperature Management (TTM) and aggres- sive post-resuscitation care dramatically improved the rate of favorable neurological recovery among patients discharged alive (81% vs. 50%). Moreover, an increase in survival could be achieved by revising and improving the quality of care, by means of periodic audits and continuous professional development cours- es (60% vs. 35%).27 Furthermore, the improvements are often described as persistent like illustrated by Venkatesh et al. for reduction in duplicate blood testing for 11 months since the begin- ning of the intervention.28 The A&F applied to emergency settings is also influenced by the geographical context. Geographic access to hospital, primary and emergency departments is an important theme for health serv- ice policy.35 Review Table 2. selected studies listed by health field, year and country. Authors Year Country Authors Year Country Cardiovascular field Infective field/ sepsis Singh K et al.6 2019 USA Hecker MT et al.13 2014 USA Bobrow BJ et al.26 2016 USA Jorgensen SCJ et al.16 2018 USA Scales DC et al.19 2016 CANADA Spencer SP et al.22 2019 USA Morrison LJ et al.25 2015 CANADA Scott RA et al.29 2014 USA Hasan DA et al.20 2019 KUWAIT Aldridge P et al.7 2017 AUSTRALIA Pellis T et al.27 2014 ITALY Khanina A et al.12 2019 AUSTRALIA Stroke Taylor KA et al.17 2018 AUSTRALIA Jauch EC et al.15 2018 USA Bentley J et al.8 2016 UK Oostema JA et al.24 2019 USA Bloos F et al.21 2017 GERMANIA Laboratory exams Other fields Reznek MA et al.14 2014 USA Venkatesh AK et al.28 2018 USA Stevens MB et al.23 2015 USA Murphy E et al.10 2015 UK Kempegowda P et al.11 2017 UK Nazerian P et al.18 2017 ITALY Bordeleau S et al.9 2015 CANADA Table 3. Different audit and feedback approaches. Multiple reports 15 Educational sessions/meetings 18 Survey online 6 Electronic methods 18 Printed materials 9 Non -co mmerc ial us e o nly [page 140] [Emergency Care Journal 2020; 16:9201] The access is influenced by many aspects that include the pres- ence of a National Health Service or private health insurance and public health coverage, various territorial organizations of health service, different geographic realities with specific critical issues, like mountain or rural contexts. In our EASY-NET sub-project we operate in a National Health Service context with many geographic realities, from urban areas to mountain areas that are difficult to access. All of these aspects have to be considered in the evaluation of the emergency settings and so in the A&F results. Other impor- tant emergency settings, often considered for A&F implementa- tion, are time depending situations, like MI and stroke: we could find them in over a third of our selected articles. Six of them (1/4 of total articles) were conducted in North America. These studies considered time-depending variables i.e. the time passed from phone call to arrival on the scene (the gold stan- dard is 15 minutes),24 interval from call receipt till the call-center operator recognized the necessity of a Telephone cardiopulmonary resuscitation, and the relative times necessary to start instructions by the operator until the by-stander performed the first chest com- pression.26 Almost all of the selected studies were focused on early recognition of the problem and early treatment. For both outcomes, A&F approach results showed an improvement; i.e.: increase in survival (60% vs. 35%), (12% vs 9.1%, p=0.02)8 even after control for potential confounders and risk measures, achievement of suc- cessful reanimation by reaching the target temperature within 6 hours (25.7% vs 9.0%, p<0.00001),25 treatment with alteplase within 60 minutes of ED arrival (1.9% vs 5.2%; p<0.01).15 The evaluation of time-depending variables is, then, a key aspect that is also fundamental for indicators of essential assistance cares, like Italian “Essential Levels of Care”,36 which include the services that the Italian National Health Service (SSN) provides to all citizens and concerns collective prevention and public health, district assistance on the territory and hospital assistance, with also A&F. The EASY-NET FVG sub-project is focused on MI and stroke and considered different time depending indicators. We can see the validity of this kind of intervention independ- ently from different A&F methods. The most used feedback meth- ods are email and frontal lessons or conferences (in about 2/3 of selected studies) and they are often combined together. The EASY- NET approach uses all these methods introducing something new: the virtual reality training approach to implement both practice and confidence. We are convinced, as many cited authors, that confi- dence combined with knowledge are a key stone, because every real change of a behavior came from a deep and personal aware- ness of the problem and a higher confidence with procedures.37,38 This is a concept applied for example in health promotion theories that should be used also in hospitals and it is at the basis of the A&F approach. A different interesting aspect to take into account is the statistical method used to evaluate the results of the interven- tion, especially the Interrupted Time Series (ITS), which is the chosen method in EASY-NET FVG sub-project. ITS analysis is arguably the strongest quasi-experimental research design, particularly useful when a randomized trial is infeasible or unethical39 and it is a valuable study design for eval- uating the effectiveness of population-level health interventions that have been implemented at a clearly defined point in time;40 thus, it’s a useful tool to measure quality improvement. The approach usually consists in the construction of a time series of population-level rates for a particular quality improvement focus, and then testing statistically how an outcome rate changes in the time periods before vs time periods after implementation of a designed intervention.39 Strengths of ITS include the ability to: i) control secular trends in data, ii) evaluate outcomes using popula- tion-level data, clear graphical presentation of results and iii) eval- uate both intended and unintended consequences of interven- tions.39 Limitations of ITS include the need for a minimum of 8 time periods before and 8 after an intervention, difficulty in analyzing the independent impact of separate components of a program that are implemented close together in time and existence of a suitable control population.39 These limitations can be the reason why ITS was applied only in 3 of our selected studies14,24,28 even if we consider ITS a better methodological approach. Other statistical methods used in the majority of our studies are: Fisher exact test, Kruskal Wallis test, Mann Whitney test,27 Chi square test, Logistic regression models [8], generalized estimating equation approach.25 All of these meth- ods have different limits: for example, a 2-period before-and-after t test can’t control secular trends in data. The unit of analysis for the comparisons often was the individ- ual patient25 while using ITS, individual-level inferences should be avoided when population-level rates are used to evaluate interven- tions.39 Despite the wide range on interventions modality and the poor comparability of the considered studies, the results of the A&F approach are encouraging and confirm the importance to imple- ment A&F both in emergency and in other clinical settings. Conclusions The primary aim of this review was to evaluate the diffusion and effectiveness of A&F in emergency departments and how the process of A&F can influence medical and, in general, health pro- fessionals’ behavior in the field of emergency care, either in the hospital or in the out of –hospital interventions. Starting with the results obtained by Rogers et al.,4 we per- formed a review of the literature from January 2014 to December 2019 and we focused on 24 publications. Based on these articles we can affirm that A&F process in EDs is almost heterogeneous and not standardized either for A&F approach or for statistical methods. Despite these limitations, all specific forms of A&F were con- sidered effective and essential for an increase of quality service in EDs in different emergency situations (sepsis, MI, cardiac arrest, stroke). References 1. Gude WT. Understanding and optimizing electronic audit and feedback to improve quality of care. PhD Thesis, University of Amsterdam, 2019. PROVIDE LINK 2. The Quality and Patient Safety Division. A practical guide to clinical audit. National quality improvement Team. August 2013. QPSD-D-029-1. Available from: https://www.hse.ie/eng/about/who/qid/measurementquality/cli nical-audit/practicalguideclaudit2013.pdf 3. Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012;6:1–227. 4. Rogers RLG, Narvaez Y, Venkatesh AK, et al. Improving emergency physician performance using audit and feedback: a systematic review. Am J Emerg Med 2015;33:1505-14. Review Non -co mmerc ial us e o nly [Emergency Care Journal 2020; 16:9201] [page 141] 5. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred report- ing items for systematic reviews and meta-analyses: the PRIS- MA statement. Int J Surg 2010;8:336–41. 6. Singh K, Devarajan R, Mohanan PP, et al. ACS QUIK Investigators. Implementation and acceptability of a heart attack quality improvement intervention in India: a mixed methods analysis of the ACS QUIK trial. Implement Sci 2019;14:12. 7. Aldridge P, Horsley E, Rosettenstein K, et al. What the FLOQ? A quality improvement project to reduce unnecessary paedi- atric respiratory viral swabs in a peripheral metropolitan hos- pital. J Paediatr Child Health 2018;54:416-9. 8. Bentley J, Henderson S, Thakore S, et al. Seeking Sepsis in the Emergency Department- Identifying Barriers to Delivery of the Sepsis 6. BMJ Qual Improv Rep 2016;5:u206760.w3983. 9. Bordeleau S, Poitras J, Marceau D, et al. Use of prothrombin complex concentrate in warfarin anticoagulation reversal in the emergency department: a quality improvement study of admin- istration delays. BMC Health Serv Res 2015;15:106. 10. Murphy E, MacGlone S, McGroarty C. A novel approach to improving coagulation sample ordering in an emergency department. BMJ Qual Improv Rep 2015;4:u204785.w2857. 11. Kempegowda P, Coombs B, Nightingale P, et al. Regular and frequent feedback of specific clinical criteria delivers a sus- tained improvement in the management of diabetic ketoacido- sis. Clin Med (Lond) 2017;17:389-94. 12. Khanina A, Cairns KA, McGloughlin S, et al. Improving sepsis care for hospital inpatients using existing medical emergency response systems. Infect Dis Health 2020;25:63-70. 13. Hecker MT, Fox CJ, Son AH, et al. Effect of a stewardship intervention on adherence to uncomplicated cystitis and pyelonephritis guidelines in an emergency department setting. PLoS One 2014;9:e87899. 14. Reznek MA, Barton BA. Improved incident reporting follow- ing the implementation of a standardized emergency depart- ment peer review process. Int J Qual Health Care 2014;26:278- 86. 15. Jauch EC, Huang DY, Gardner AJ, Blum JL. Strategies for improving outcomes in the acute management of ischemic stroke in rural emergency departments: a quality improvement initiative in the Stroke Belt. Open Access Emerg Med 2018;10:53-9. 16. Jorgensen SCJ, Yeung SL, Zurayk M, et al. A. Leveraging Antimicrobial Stewardship in the Emergency Department to Improve the Quality of Urinary Tract Infection Management and Outcomes. Open Forum Infect Dis 2018;5:ofy101. 17. Taylor KA, Durrheim DN, Merritt T, et al. Multidisciplinary analysis of invasive meningococcal disease as a framework for continuous quality and safety improvement in regional Australia. BMJ Open Qual 2018;7:e000077. 18. Nazerian P, Vanni S, Fanelli A, et al. Appropriate use of labo- ratory test requests in the emergency department: a multilevel intervention. Eur J Emerg Med 2019;26:205-11. 19. Scales DC, Golan E, Pinto R, et al. Strategies for Post-Arrest Resuscitation Care Network. Improving Appropriate Neurologic Prognostication after Cardiac Arrest. A Stepped Wedge Cluster Randomized Controlled Trial. Am J Respir Crit Care Med 2016;194:1083-91. 20. Hasan DA, Drennan J, Monger E, et al. Dispatcher assisted cardiopulmonary resuscitation implementation in Kuwait: A before and after study examining the impact on outcomes of out of hospital cardiac arrest victims. Med (Baltimore) 2019;98:e17752. 21. Bloos F, Rüddel H, Thomas-Rüddel D, et al. MEDUSA study group. Effect of a multifaceted educational intervention for anti-infectious measures on sepsis mortality: a cluster random- ized trial. Intensive Care Med 2017;43:1602-12. 22. Spencer SP, Karsies T. Audit-and-Feedback and Workflow Changes Improve Emergency Department Care of Critically III Children. Pediatr Qual Saf 2019;4:e128. 23. Stevens MB, Hastings SN, Powers J, et al. Enhancing the Quality of Prescribing Practices for Older Veterans Discharged from the Emergency Department (EQUiPPED): Preliminary Results from Enhancing Quality of Prescribing Practices for Older Veterans Discharged from the Emergency Department, a Novel Multicomponent Interdisciplinary Quality Improvement Initiative. J Am Geriatr Soc 2015;63:1025-9. 24. Oostema JA, Chassee T, Baer W, et al. Brief Educational Intervention Improves Emergency Medical Services Stroke Recognition. Stroke 2019;50:1193-200. 25. Morrison LJ, Brooks SC, Dainty KN, et al. Strategies for Post- Arrest Care Network. Improving use of targeted temperature management after out-of-hospital cardiac arrest: a stepped wedge cluster randomized controlled trial. Crit Care Med 2015;43:954-64. 26. Bobrow BJ, Spaite DW, Vadeboncoeur TF, et al. Implementation of a Regional Telephone Cardiopulmonary Resuscitation Program and Outcomes After Out-of-Hospital Cardiac Arrest. JAMA Cardiol 2016;1:294-302. 27. Pellis T, Sanfilippo F, Roncarati A, et al. A 4-year implementa- tion strategy of aggressive post-resuscitation care and temper- ature management after cardiac arrest.Resuscitation. 2014 Sep;85(9):1251-6. 28. Venkatesh AK, Hajdasz D, Rothenberg C, et al. Reducing Unnecessary Blood Chemistry Testing in the Emergency Department: Implementation of Choosing Wisely. Am J Med Qual 2018;33:81-5. 29. Scott RA, Oman KS, Makic MB, et al. Reducing indwelling urinary catheter use in the emergency department: a successful quality-improvement initiative. J Emerg Nurs 2014;40:237-44. 30. van de Ridder JM, Stokking KM, McGaghie WC, ten Cate OT. What is feedback in clinical education?. Med Educ 2008;42:189-97. 31. Thomas JD, Arnold RM. Giving feedback. J Palliat Med 2011;14:233-9. 32. Gude WT, Brown B, van der Veer SN, et al. Clinical perfor- mance comparators in audit and feedback: a review of theory and evidence. Implement Sci 2019;14:39. 33. Ullman AJ, Ray-Barruel G, Rickard CM, Cooke M. Clinical audits to improve critical care: Part 1 Prepare and collect data. Aust Crit Care 2018;31:101-5. 34. Ray-Barruel G, Ullman AJ, Rickard CM, Cooke M. Clinical audits to improve critical care: Part 2: Analyse, benchmark and feedback. Aust Crit Care 2018;31:106-9. 35. Fone DL, Christie S, Lester N. Comparison of perceived and modelled geographical access to accident and emergency departments: a cross-sectional analysis from the Caerphilly Health and Social Needs Study Int J Health Geogr 2006;5:16. 36. Decree by the President of the Italian Council of Ministers. [Definizione e aggiornamento dei livelli essenziali di assisten- za, di cui all'articolo 1, comma 7, del decreto legislativo 30 dicembre 1992, n. 502. (17A02015). (G.U. Serie Generale, n. 65 del 18 marzo 2017).] Available from: http://www.trovanorme.salute.gov.it/norme/dettaglioAtto?id= 58669&completo=true [Website in Italian]. 37. Boling B, Hardin-Pierce M. The effect of high-fidelity simula- Review Non -co mmerc ial us e o nly [page 142] [Emergency Care Journal 2020; 16:9201] tion on knowledge and confidence in critical care training: An integrative review. Nurse Educ Pract 2016;16:287-93. 38. Pulford BD, Colman AM, Buabang EK, Krockow EM. The persuasive power of knowledge: Testing the confidence heuris- tic. J Exp Psychol Gen 2018;147:1431-44. 39. Penfold RB, Zhang F. Use of Interrupted Time Series Analysis in Evaluating Health Care Quality Improvements. Acad Pediatr 2013;13:S38-44. 40. Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol 2017;46:348-55. Review Non -co mmerc ial us e o nly