Hrev_master [page 72] [Emergency Care Journal 2023; 19:11443] Emergency Care Journal 2023; volume 19:11443 Abstract The study aimed to validate the Manchester Triage System in a hospital setting using data for short- and medium-term death rates. A prospective observational study was conducted at the Emergency Department of Merano Hospital for two years. The discriminatory ability of MTS was tested using AUROCs and contingency tables, reporting sensitivity and specificity levels for each study outcome. A total of 98,443 patients were enrolled, 237 of whom died within 72h; 422 patients died within seven days, and 1025 died within 30 days. The MTS demonstrated excellent discriminatory ability, reporting AUROC values of 0.890 for death within 72h, 0.853 for death within seven days, and 0.781 for death within 30 days. A sen- sitivity of 87.7% and a specificity of 79.4% were reported for death at 72h, while a sensitivity of 69.6% and a specificity of 79.8% were reported for death at 30 days. The MTS has proven to be a good triage system capable of accurately identifying patients who are at risk of death in the short or medium term. Introduction The progressive increase in the number of requests for evalua- tion in hospitals’ Emergency Departments (EDs) after the COVID- 19 pandemic, given an ever-decreasing number of available health- care resources, makes triage one of the most important components of the emergency medical system.1,2 At present, internationally val- idated triage systems (e.g., the Manchester Triage System, the Emergency Severity Index, and the South African Triage Scale) that can be compared between different regions and nations remain rare. Although there are recommendations supporting the use of globally standardized triage systems, many European countries today have adopted autonomous systems created according to local needs. These systems often lack scientific validation, making it impossible to perform comparisons.3-6 In addition to being an expensive choice, given the significant amounts of resources required for the creation and design of new triage tools, these sys- tems often merely simulate or integrate triage methodologies already widely developed in validated systems, hence becoming a pure relocation exercise that limits the development of a crucial aspect of the ED. A recent review of the literature on the perfor- mance of validated triage systems showed that there were no sub- stantial differences in the ability to prioritize patients between the various systems and emphasized that regardless of the instrument used, performance remains comparable.3 The decision not to import one of these already structured systems to invest in local systems has led to the failure to evolve international triage sys- tems, thus leaving them virtually unchanged since their creation in Correspondence: Arian Zaboli, Department of Emergency Medicine, Hospital of Merano-Meran (SABES-ASDAA), via Rossini 5, 39012, Merano (BZ), Italy. E-mail: zaboliarian@gmail.com Key words: triage; Manchester triage system; emergency medicine; nursing; nurse triage. Contributions: AZ, GT, conceptualization, methodology, investiga- tion, formal analysis, data curation, writing-original draft prepara- tion; SS, methodology, investigation, writing-original draft prepara- tion; AC, NP: original draft preparation; AB: supervision; NP: data curation; AG: investigation; FB: supervision, original draft prepara- tion, review, and editing. Conflict of interest: the authors declare no potential conflict of inter- est, and all authors confirm accuracy. Ethics approval: the study was approved by the local ethics commit- tee (Comitato etico per la sperimentazione clinica, Azienda Sanitaria dell’Alto Adige, Bolzano, Italia, approval number 95-2019) and was conducted in accordance with the Declaration of Helsinki regarding the Ethical Principles for Medical Research Involving Human Subjects. Informed consent: all patients participating in this study signed a written informed consent form for participating in this study. Patient consent for publication: written informed consent was obtained from a legally authorized representative(s) for anonymized patient information to be published in this article. Availability of data and materials: all data generated or analyzed during this study are included in this published article. Received: 2 May 2023. Accepted: 30 June 2023. Early access publication: 11 June 2023. This work is licensed under a Creative Commons Attribution 4.0 License (by-nc 4.0). ©Copyright: the Author(s), 2023 Licensee PAGEPress, Italy Emergency Care Journal 2023; 19:11443 doi:10.4081/ecj.2023.11443 Publisher's note: all claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher. Italian validation of the Manchester Triage System towards short-term mortality: a prospective observational study Arian Zaboli,1,2 Serena Sibilio,1,2 Alessandro Cipriano,3 Naria Park,4 Antonio Bonora,5 Norbert Pfeifer,1,2 Alberto Giudiceandrea,1,2 Francesco Brigo,2,6 Gianni Turcato7 1Department of Emergency Medicine, Hospital of Merano-Meran (SABES-ASDAA), Merano-Meran, Italy; 2Lehrkrankenhaus der Paracelsus Medizinischen Privatuniversität, Salzburg, Austria; 3Medicina d’Urgenza e Pronto Soccorso Azienda Ospedaliera Universitaria Pisana, Pisa, Italy; 4Medicina d’Urgenza Universitaria Azienda Ospedaliera Universitaria Pisana, Pisa, Italy; 5Emergency Department, University of Verona, Verona, Italy; 6Department of Neurology, Hospital of Merano-Meran (SABES-ASDAA), Merano-Meran, Italy; 7Department of Internal Medicine, Intermediate Care Unit, Hospital Alto Vicentino (AULSS-7), Santorso, Italy Non -co mmerc ial us e o nly the 1990s.3-5 One of the most widespread and globally used sys- tems is the Manchester Triage System (MTS), a system that classi- fies patients into five priority levels in association with 53 symp- tom-specific flow charts.7-9 General information on the ability of MTS to correctly stratify patients is limited, and few studies are validating the performance of the system against short- or medi- um-term mortality.4,8,10 Moreover, validation studies often used outcomes other than mortality and are often subject to the subjec- tivity of the evaluators (e.g., a pool of experts).4,8,10,11 Currently, there is no single triage system in Italy that is the standard for EDs in different regions; as a result, each department employs a contextual method with limited supporting scientific evidence. Moreover, the use of the MTS in Italian emergency and urgent care settings has not yet been examined. The present study aimed to validate the MTS against short-term (72 hours) and medi- um-term (7-30 days) mortality in an Italian context. Materials and Methods Setting The present single-center prospective observational study was conducted at the ED of the Merano General Hospital from 1 January 2021 to 31 December 2022. The ED under study has been using the MTS as a triage system since 2014. Triage is performed by two dedicated nurses during the day shift (08.00-20.00) and by one nurse during the night shift (20.00-08.00). The nurses perform- ing triage in the ED must have at least two years of experience in a critical area setting; they must have completed a two-day course on the MTS method and must have undergone a six-month coach- ing period with an experienced triage nurse. Manchester Triage System The MTS is based on stratifying patients into five priority lev- els, where code 1 (red, immediate) stipulates a waiting time until medical attention of 0 minutes; code 2 (orange, urgency) defines a target waiting time of 10 minutes; code 3 (yellow, urgency) speci- fies a target waiting time of 60 minutes; code 4 (green, deferrable urgency) specifies a waiting time of 120 minutes, and code 5 (blue, non-urgent) specifies a waiting time of 240 minutes.11 In the case of a longer waiting time than that established by the priority code, the triage nurse must perform a new assessment of the patient to check their condition and consequently confirm or change the pre- viously assigned priority code. The classification system of the MTS is based on 53 specific symptom diagrams (e.g., chest pain, palpitations, or wounds), where a different flow chart is associated with each symptom. Each flow chart has rapid indicators (specific symptom questions) to be scrolled through that define the level of priority from most urgent to least urgent. If no indicators are positive, the patient is classified with a priority code of 5. At the ED under study, the 3rd edition of MTS was used, using the German text.10 Patients and variables All patients admitted to the ED during the study period were enrolled. Only patients that were non-residents (e.g., tourists) of the province under study were excluded due to the impossibility of reconstructing the subsequent outcome. For all patients who con- sented to participate, their data, completed triage forms, and ED medical records were collected. All information gathered was sub- sequently entered anonymously into an electronic database. Outcome The primary outcome of the study is composed of sensitivity, specificity, positive predictive value, negative predictive value, and the area under the ROC curve (AUROC) of the triage assess- ment compared to the 72-hour mortality. The secondary outcomes are composed of sensitivity, specificity, positive predictive value, negative predictive value, and the area under the AUROC of the triage assessment compared to seven- and 30-days mortality. The outcome was reconstructed by manual re-evaluation of medical records in the case of hospitalized patients or via the register office. Ethical considerations The study was approved by the local ethics committee (Comitato etico per la sperimentazione clinica, Azienda Sanitaria dell’Alto Adige, Bolzano, Italia, approval number 95-2019) and was conducted in accordance with the Declaration of Helsinki regarding the Ethical Principles for Medical Research Involving Human Subjects. Statistical analysis Continuous variables were described as the mean or interquar- tile range. The comparisons between different patient classes were performed with the Mann-Whitney test or the Kruskal-Wallis test as appropriate. Categorical variables were expressed as a percent- age of the number of events out of the total, and comparisons between the patient classes were performed with Fisher’s exact test or the Chi-square test as appropriate. The validity of the MTS was assessed by casting the data into 2×2 contingency tables to compare short-term (72 hours and seven days) and medium-term (30 days) mortality with MTS codes, where priority codes 1 (red) and 2 (orange) were combined as high priority codes, and priority codes 3 (yellow), 4 (green), and 5 (blue) was considered as low priority codes. This choice was made in agreement with previous studies. The 2×2 contingency tables were used to examine sensitivity (the ability of the MTS to correct- ly identify persons with high priority codes as being at risk of death in the short to medium term), specificity (the ability of the MTS to identify patients with low priority codes who did not die in the short to medium term), negative predictive value (NPV) (the prob- ability of not dying in the short to medium term when the MTS assigned a low priority code), and positive predictive value (PPV) (the probability of dying in the short to medium term when the MTS assigned one of the high priority codes). The five priority levels of the MTS were also assessed via the AUROC. Subsequently, analyses were performed via the ROCs for subgroups of patients according to age (≥75 years, ≥65 years, >35 and <65 years, and ≤35 years), entry problem (medical-surgical entry symptoms, thus excluding fast-track), mode of arrival (out- of-hospital service or self-care), and conclusion after evaluation (admission or discharge). All results were described with their 95% confidence intervals (95% CI). Statistical analysis was performed using STATA 16.0 (StataCorp, College Station, TX, USA). Results The number of patients admitted between 2021 and 2022 was 120,258 (Figure 1). There were 98,443 patients enrolled in the study, of whom 0.4% (367/98,443) were code 1 (red), 4.7% (4636/98,388) were code 2 (orange), 5.6% (15,381/98,443) were code 3 (yellow), 69.5% (68,388/98,443) were code 4 (green) and Article [Emergency Care Journal 2023; 19:11443] [page 73] Non -co mmerc ial us e o nly 9.8% (6971/98,443) were code 5 (blue). The characteristics of the patients are shown in Table 1. Patients with higher codes were older and came to the ED more often by ambulance, by an emergency physician, or by heli- copter. Patients with higher codes were more likely to be placed on a stretcher or in a wheelchair in the ED and to have more interven- tions performed in triage (ECG, blood sampling, or venous access placement). Patients assigned higher codes experienced more hos- pitalizations and more short- and medium-term deaths. In contrast, patients with lower codes were younger and were more likely to have reached the ED independently. In addition, patients with lower codes had fewer altered vital signs in triage and required fewer admissions. Overall, the percentages of patients who died at 72h, seven days, and 30 days were 0.2% (237/98,443), 0.4% (422/98,443), and 1.0% (1025/98,443), respectively. The median length of stay (LOS) in the ED was 1.7 hours (95% CI: 0.9-2.9), while the median triage time was 3.3 minutes (95% CI: 1.8-6.6). The discriminatory ability of the MTS against 72-h, 7-day, and 30-day mortality is shown in Figure 2. The AUROC values varied from 0.781 (95% CI 0.765-0.798) for 30-day mortality to 0.853 (95% CI 0.831-0.875) for 7-day mortality and finally to 0.890 (95% CI 0.865-0.916) for 72-hour mortality, performance for each cut-point of the ROC and each outcome is reported in the Supplementary Table 1. Considering only patients assigned to the areas of medicine or surgery, the discriminatory ability scores of the MTS for 72-h, 7-day, and 30-day mortality were 0.877 (CI 0.851-0.952), 0.837 (CI 0.814-0.860), and 0.770 (CI 0.753-0.787), respectively. The sensitivity of the MTS and thus the ability to accurately classify patients as high-priority codes regarding mid- and short- term mortality was very high, ranging from 87.7% for death at 72 Article Figure 1. Flow-chart of patients enrolled in the study. Figure 2. AUROC of MTS for the three selected outcomes. [page 74] [Emergency Care Journal 2023; 19:11443] Non -co mmerc ial us e o nly hours to 69.6% for death within 30 days. The specificity of the MTS and thus the assignment of low-priority codes to patients who subsequently did not have the study outcome demonstrated excel- lent performance, with values ranging from 79.4% for death at 72 h to 79.8% for death at 30 days. Subsequently, subgroup AUROC values were calculated in comparisons of short- and medium-term mortality (Table 2). In the comparisons of the groups of older adult patients (≥ 65 years and ≥ 75 years) in all outcomes considered, the performance of the sys- tem was excellent, with AUROC values ranging from 0.673 to 0.865. The best performance was reported in patients aged < 35 years, with AUROC values ranging from 0.942 to 0.946, demon- strating nearly perfect discriminatory ability (Table 3). The worst performance was reported in the subgroup of patients hospitalized after ED evaluation, with AUROC values ranging from 0.673 to 0.779; this was still considered a good per- formance for this subgroup (Table 3). Discussion The current study using a large cohort of patients from an Italian ED represents the first national-level validation of the MTS by examining the performance of the system regarding short-term mortality. Italy currently has no objectively validated national triage system. Instead, several different systems have been created over the years according to local needs, and these are often lacking external or even internal validation, and very few have been pub- lished.12 In addition, there is considerable subjectivity in many of these systems, as the triage nurses do not always perform their assessment using objective criteria, but rather base their decisions on prior knowledge and experience, a concept that is largely out- dated in the triage setting.10,12-14 This study demonstrated the remarkable ability of the MTS in stratifying patients accessing the ED, identifying those at risk of death in the short and medium term. Thus, the study suggests that given the difficulty of unifying triage systems in Italy, the MTS could be implemented in all national EDs given its worldwide dis- Article Table 1. Anamnestic and clinical characteristics of patients enrolled in the study divided according to the priority code assigned in triage. Heart Rate* was collected in 38,683 Patients; Respiratory Rate was collected in 29,967 Patients; Oxygen Saturation* was collected in 39,783 Patients; Systolic blood pressure was collected in 33,361 Patients; Temperature* was collected in 65,859 Patients. Variables Blue & Green priority code Yellow priority code Orange & Red priority code P Patients, n (%) 78,059 (79.3) 15,381 (15.6) 5003 (5.1) Age, years, mean (SD) 55.8 (25.1) 50.5 (28.2) 58.1 (25.7) <0.001 Modality of arrival in ED, n (%) Autonomous 66,895 (85.7) 9,043 (58.8) 2,161 (43.2) <0.001 Ambulance 10,517 (13.5) 5,420 (35.2) 1,869 (37.4) <0.001 Emergency physician 580 (0.7) 808 (5.3) 856 (17.1) <0.001 Helicopter 67 (0.1) 110 (0.7) 117 (2.3) <0.001 After triage positioned, n (%) Walk 66,818 (85.6) 7,736 (50.3) 1,276 (25.5) <0.001 Wheelchair 9,367 (12.0) 4,691 (30.5) 1,085 (21.7) <0.001 Stretcher 1,873 (2.4) 2,953 (19.2) 2,641 (52.8) <0.001 Vital parameters Heart Rate*, median (IQR) 85 (74-96) 85 (74-100) 90 (75-110) <0.001 Respiratory Rate*, median (IQR) 16 (14-17) 16 (15-18) 18 (16-25) <0.001 Oxygen Saturation*, median (IQR) 98 (97-99) 98 (96-99) 97 (93-98) <0.001 Systolic blood pressure*, mean (SD) 137.6 (23.4) 137.7 (25.4) 136.7 (29.6) 0.128 Temperature*, median (IQR) 36.2 (36.0-36.5) 36.3 (36.0-36.8) 36.8 (36.4-36.8) <0.001 Performances done in triage by the nurse, n (%) Venous access 6,731 (8.6) 3,984 (25.9) 1,978 (39.5) <0.001 ECG 5,398 (6.9) 2,676 (17.4) 1,589 (31.8) <0.001 Blood Sampling 6,700 (8.6) 3,793 (24.7) 1,887 (37.7) <0.001 Area of the declared symptoms, n (%) Internal medicine 17,140 (22.0) 5,133 (33.4) 3,277 (65.5) <0.001 Surgery 13,853 (17.7) 2,876 (18.7) 437 (8.7) <0.001 Orthopaedics 22,736 (29.1) 2,794 (18.2) 130 (2.6) <0.001 Urological 1,977 (2.5) 891 (5.8) 73 (1.5) <0.001 Otolaryngology & dentistry 6,588 (8.4) 187 (1.2) 5 (0.1) <0.001 Ophthalmology 3,851 (4.9) 424 (2.8) 32 (0.6) <0.001 Paediatrics 6,519 (8.4) 1,520 (9.9) 216 (4.3) <0.001 Gynaecological 3,000 (3.8) 986 (6.4) 787 (15.7) <0.001 Psychiatric 442 (0.6) 525 (3.4) 45 (0.9) <0.001 Dermatology 1,953 (2.5) 45 (0.3) 1 (0.0) <0.001 Hospitalised, n (%) 4,094 (5.2) 3,454 (22.5) 2,562 (51.2) <0.001 ED abandonment before medial evaluation, n (%) 2400 (3.1) 132 (0.9) 29 (0.6) <0.001 Death within 72h, n (%) 29 (0.04) 51 (0.3) 157 (3.1) <0.001 Death within 7 days, n (%) 76 (0.1) 105 (0.7) 241 (4.8) <0.001 Death within 30 days, n (%) 311 (0.4) 289 (1.9) 425 (8.5) <0.001 [Emergency Care Journal 2023; 19:11443] [page 75] Non -co mmerc ial us e o nly semination and previously published validation studies.10,13,14 In addition to being an objective system with good predictive ability and prioritization capability on objective data such as short-term mortality, the MTS could allow broad comparisons across the EDs among Western countries. The present study has important consequences for clinical practice. First, the study yielded high AUROC values in the short- and medium-term mortality comparisons. A previous validation study of the MTS performed in German-speaking European areas by Graff et al. reported an AUROC of 0.613 for 30-day mortality, and the result was confirmed by subsequent research evaluating the performance of the MTS based on selected flow charts, with AUROC values ranging from 0.682 to 0.834 for in-hospital mor- tality.14,15 The differences in results between the present and previ- ous studies may lie in the improvements made in EDs under inves- tigation (such as, for example, daily triage auditing practices), demonstrating that continuous and consistent attention can lead to higher performance and better outcomes for patients.16 The second novelty of the study in view of the current interna- tional literature is the choice of objective outcomes (72-h, 7-day, and 30-day mortality) for validation. In previous research, the out- comes chosen were generally subjective measures such as confir- mation of code appropriateness as determined by a pool of experts or the use of reference standards.13 Reference standards, however, Article Table 2. 2x2 contingency tables divided according to the three study outcomes and the priority code assigned in triage, where low priority code considers patients with code 5 (blue) and code 4 (green) and high priority code considers patients with code 3 (yellow), code 2 (orange) and code 1 (red). Sensibility, specificity, PPV and NPV with respective 95% confidence intervals were calculated for each con- tingency table. Death within 72h No Yes Sensibility Specificity PPV NPV Low priority code 78.030 29 87.7% (82.9-91.6) 79.4% (79.2-79.7) 1.0% (0.9-1.1) 99.9% (99.9-100.0) High priority code 20.176 208 Death within 7 days No Yes Sensibility Specificity PPV NPV Low priority code 77.983 76 81.9% (81.7-82.2) 79.5% (79.3-79.8) 1.7% (1.6-1.8) 99.9% (99.8-99.9) High priority code 20.038 346 Death within 30 days No Yes Sensibility Specificity PPV NPV Low priority code 77.748 311 69.6% (69.4-69.9) 79.8% (79.5-80.1) 3.5% (3.4-3.6) 99.6% (99.6-99.6) High priority code 19.670 714 Table 3. AUROC of MTS performance calculated for subgroups of patients divided according to the three study outcomes. Death within 72 h Subgroups AUROC 95% Confidence interval ≥ 75 years 0.863 0.835-0.892 ≥ 65 years 0.865 0.839-0.892 Between 35 & 65 years 0.890 0.866-0.913 <35 years 0.946 0.843-1.000 Internal Medicine & Surgery 0.876 0.852-0.900 Ambulance 0.802 0.762-0.841 Autonomous 0.865 0.806-0.924 Hospitalised 0.779 0.745-0.814 Discharged 0.863 0.810-0.917 Death within 7 days ≥ 75 years 0.809 0.784-0.834 ≥ 65 years 0.822 0.799-0.844 Between 35 & 65 years 0.853 0.832-0.873 <35 years 0.946 0.843-1.000 Internal Medicine & Surgery 0.836 0.815-0.858 Ambulance 0.758 0.727-0.789 Autonomous 0.796 0.744-0.849 Hospitalised 0.741 0.713-0.769 Discharged 0.792 0.745-0.840 Death within 30 days ≥ 75 years 0.742 0.724-0.761 ≥ 65 years 0.752 0.735-0.768 Between 35 & 65 years 0.781 0.766-0.796 <35 years 0.942 0.887-0.997 Internal Medicine & Surgery 0.770 0.753-0.786 Ambulance 0.685 0.664-0.706 Autonomous 0.730 0.699-0.760 Hospitalised 0.673 0.652-0.695 Discharged 0.702 0.675-0.729 [page 76] [Emergency Care Journal 2023; 19:11443] Non -co mmerc ial us e o nly are created by a panel of experts as study outcomes for validating triage systems, but at present, there are no data available regarding their actual effectiveness or usefulness.13,17 The choice to use out- comes such as mortality, in contrast to previous studies that select- ed objective and subjective outcomes simultaneously, is supported by the absence of gold standards for the evaluation of triage sys- tems.17,18 Furthermore, a review by Kuriyama et al. concerning the validity of five-level triage systems suggested the use of outcomes different from reference standards or expert consensus, precisely because of the inherent subjectivity and the limited number of patients potentially re-evaluable by the expert pool.17 However, it is important to point out the weaknesses of the objective outcomes considered; in fact, short- and medium-term mortality is undoubt- edly influenced by the choices made and the events following the triage assessment.17 Furthermore, as reported by Challen, triage is not meant to diagnose a disease or predict death, but rather is meant to characterize urgency.18 Objective outcomes such as death may not be appropriate for all patients (e.g., patients with severe pain as the result of a limb fracture will not be at risk of death but will need a higher code given the need for rapid pain manage- ment).18 Third, considering subgroups of patients categorized by age, the MTS has been shown to perform well in both younger and older patient groups. Due to the increasing older adult population accessing the ED and their inherent frailty, research in recent years has focused on evaluating this class of patients.19,20 Brouns et al. in a retrospective study of the MTS evaluated the performance of the triage system toward in-hospital mortality by dividing subjects into older patients (>64 years) and adult patients (18-64 years).21 The results demonstrated a worse performance of the MTS for older patients (AUROC 0.71; 95%CI 0.68 - 0.74) compared to adults (AUROC 0.79; 95%CI 0.72 - 0.85), although indicating a good overall ability of the triage system.21 The worse performance of the MTS in this group of patients may be related to the high number of comorbidities typical in older patients, a factor that the MTS does not consider and that may worsen or alter the manifestation in the short term, resulting in unexpected outcomes that are difficult to detect in triage. Given the overall aging demographic and the increase of multi-pathology patients accessing the ED, it is neces- sary to study and understand how to integrate specific assessments within a triage system.22,23 The system should consider the patient’s innate comorbidity quota, thus allowing the MTS to optimally classify older patients. Despite this limitation, the good perfor- mance against mortality supports the use of the MTS and argues for its future evolution rather than its abandonment. The present study has several limitations. First, the single-cen- tre nature of the study limits the generalisation of the results. Second, the non-consideration of more subjective endpoints such as the assessment of code adequacy may have biased the results. Given the number of patients, it was impractical to manually re- evaluate all relevant records. Third, the correct application of the MTS was not assessed; rather, this was monitored daily through direct feedback within the ED under study, however, it is not pos- sible to be certain about the correct application of MTS.16 Fourth, the version used for the implementation of MTS is based on the German book and not on the original one. Fifth, the outcome used is limiting for the complete evaluation of a triage system and there- fore the results cannot completely represent the real functioning of MTS. Conclusions This is the first validation study of the MTS performed against short- and medium-term death. The system demonstrated excellent performance within an Italian hospital setting. The identification of patients at risk of death in the short and medium term was quite accurate, even for the most challenging groups such as older adult patients. If these results were subsequently confirmed by further studies, the use of a validated triage tool throughout the Italian country could be recommended. References 1. Zaboli A, Brigo F, Sibilio S, et al. The impact of COVID-19 pandemic on the urgency of patients admitted to the emergency department. Int Emerg Nurs 2022;65:101229. 2. Gilbert A, Brasseur E, Petit M, Donneau AF, D’Orio V, Ghuysen A. Advanced triage to redirect non-urgent Emergency Dœepartment visits to alternative care centers: the PERSEE algorithm. Acta Clin Belg 2022;77:571-8. 3. Hinson JS, Martinez DA, Cabral S, et al. Triage Performance in Emergency Medicine: A Systematic Review. Ann Emerg Med 2019;74:140-52. 4. Zachariasse JM, van der Hagen V, Seiger N, et al. Performance of triage systems in emergency care: a systematic review and meta-analysis. BMJ Open 2019;9:e026471. 5. Christ M, Grossmann F, Winter D, Bingisser R, Platz E. Modern triage in the emergency department. Dtsch Arztebl Int 2010;107:892-8. 6. Garrafa E, Levaggi R, Miniaci R, Paolillo C. When fear back- fires: Emergency department accesses during the Covid-19 pandemic. Health Policy 2020;124:1333-9. 7. Brigo F, Zaboli A, Rinaldi F, et al. The Manchester Triage System’s performance in clinical risk prioritisation of patients presenting with headache in emergency department: A retro- spective observational study. J Clin Nurs 2022;31:2553-61. 8. Azeredo TR, Guedes HM, Rebelo de Almeida RA, et al. Efficacy of the Manchester Triage System: a systematic review. Int Emerg Nurs 2015;23:47-52. 9. Zaboli A, Ausserhofer D, Sibilio S, et al. Nurse triage accuracy in the evaluation of syncope according to European Society of Cardiology guidelines. Eur J Cardiovasc Nurs 2022;21:280-6. 10. Gräff I, Latzel B, Glien P, et al. Validity of the Manchester Triage System in emergency patients receiving life-saving intervention or acute medical treatment-A prospective obser- vational study in the emergency department. J Eval Clin Pract 2019;25:398-403. 11. Parenti N, Reggiani ML, Iannone P, et al. A systematic review on the validity and reliability of an emergency department triage scale, the Manchester Triage System. Int J Nurs Stud 2014;51:1062-9. 12. Canzan F, Allegrini E, Mezzalira E, et al. Il processo decision- ale dell’infermiere nell’attribuzione del codice di triage: uno studio qualitativa descrittivo [Nurse decision making in triage cose assignment: a qualitative descriptive study]. Assist Inferm Ric 2022;41:15-22. 13. Zachariasse JM, Seiger N, Rood PP, et al. Validity of the Manchester Triage System in emergency care: A prospective observational study. PLoS One 2017;12:e0170811. 14. Gräff I, Goldschmidt B, Glien P, et al. The German Version of Article [Emergency Care Journal 2023; 19:11443] [page 77] Non -co mmerc ial us e o nly the Manchester Triage System and its quality criteria—first assessment of validity and reliability. PLoS One 2014;9:e88995. 15. Brutschin V, Kogej M, Schacher S, et al. The presentational flow chart “unwell adult” of the Manchester Triage System- Curse or blessing? PLoS One 2021;16:e0252730. 16. Zaboli A, Sibilio S, Magnarelli G, et al. Daily triage audit can improve nurses’ triage stratification: A pre-post study. J Adv Nurs 2023;79:605-15. 17. Kuriyama A, Urushidani S, Nakayama T. Five-level emergen- cy triage systems: variation in assessment of validity. Emerg Med J 2017;34:703-10. 18. Challen K. How good is triage, and what is it good for? Emerg Med J 2017;34:702. 19. van Dam CS, Hoogendijk EO, Mooijaart SP, et al. A narrative review of frailty assessment in older patients at the emergency department. Eur J Emerg Med 2021;28:266-76. 20. Elliott A, Hull L, Conroy SP. Frailty identification in the emer- gency department-a systematic review focussing on feasibility. Age Ageing 2017;46:509-13. 21. Brouns SHA, Mignot-Evers L, Derkx F, et al. Performance of the Manchester triage system in older emergency department patients: a retrospective cohort study. BMC Emerg Med 2019;19:3. 22. Elliott A, Taub N, Banerjee J, et al. Does the Clinical Frailty Scale at Triage Predict Outcomes From Emergency Care for Older People? Ann Emerg Med 2021;77:620-7. 23. Salvi F, Morichi V, Grilli A, et al. Screening for frailty in elder- ly emergency department patients by using the Identification of Seniors At Risk (ISAR). J Nutr Health Aging 2012;16:313- 8. Article [page 78] [Emergency Care Journal 2023; 19:11443] Online supplementary material: Table 1. Sensitivity, specificity and likelihood ratio values for each cut-point of the AUROC curves for the three study outcomes. Non -co mmerc ial us e o nly