Microsoft Word - 11443-ecj_ea.docx Emergency Care Journal Official Journal of the Academy of Emergency Medicine and Care (AcEMC) eISSN 2282-2054 https://www.pagepressjournals.org/index.php/ecj/index Publisher's Disclaimer. E-publishing ahead of print is increasingly important for the rapid dissemination of science. The Early Access service lets users access peer-reviewed articles well before print / regular issue publication, significantly reducing the time it takes for critical findings to reach the research community. These articles are searchable and citable by their DOI (Digital Object Identifier). The Emergency Care Journal is, therefore, e-publishing PDF files of an early version of manuscripts that undergone a regular peer review and have been accepted for publication, but have not been through the typesetting, pagination and proofreading processes, which may lead to differences between this version and the final one. The final version of the manuscript will then appear on a regular issue of the journal. E-publishing of this PDF file has been approved by the authors. Emerg care J 2023 [Online ahead of print] To cite this Article: Zaboli A, Sibilio S, Cipriano A, et al. Italian validation of the Manchester Triage System towards short-term mortality: a prospective observational study. Emerg Care J doi: 10.4081/ecj.2023.114433 ©The Author(s), 2023 Licensee PAGEPress, Italy Note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries should be directed to the corresponding author for the article. All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher. Italian validation of the Manchester Triage System towards short-term mortality: a prospective observational study Arian Zaboli,1,2 Serena Sibilio,1,2 Alessandro Cipriano,3 Naria Park,4 Antonio Bonora,5 Norbert Pfeifer,1,2 Alberto Giudiceandrea,1,2 Francesco Brigo,2,6 Gianni Turcato7 1Department of Emergency Medicine, Hospital of Merano-Meran (SABES-ASDAA), Merano- Meran, Italy; 2Lehrkrankenhaus der Paracelsus Medizinischen Privatuniversität, Salzburg, Austria; 3Medicina d’Urgenza e Pronto Soccorso Azienda Ospedaliera Universitaria Pisana, Pisa, Italy; 4Medicina d’Urgenza Universitaria Azienda Ospedaliera Universitaria Pisana, Pisa, Italy; 5Emergency Department, University of Verona, Verona, Italy; 6Department of Neurology, Hospital of Merano-Meran (SABES-ASDAA), Merano-Meran, Italy; 7Department of Internal Medicine, Intermediate Care Unit, Hospital Alto Vicentino (AULSS-7), Santorso, Italy Correspondence: Arian Zaboli, Department of Emergency Medicine, Hospital of Merano-Meran (SABES-ASDAA), via Rossini 5, 39012, Merano (BZ), Italy. E-mail: zaboliarian@gmail.com Key words: triage; Manchester triage system; emergency medicine; nursing; nurse triage. Contributions: AZ, GT, conceptualization, methodology, investigation, formal analysis, data curation, writing-original draft preparation; SS, methodology, investigation, writing-original draft preparation; AC, NP: original draft preparation; AB: supervision; NP: data curation; AG: investigation; FB: supervision, original draft preparation, review, and editing. Conflict of interest: the authors declare no potential conflict of interest, and all authors confirm accuracy. Ethics approval: the study was approved by the local ethics committee (Comitato etico per la sperimentazione clinica, Azienda Sanitaria dell’Alto Adige, Bolzano, Italia, approval number 95- 2019) and was conducted in accordance with the Declaration of Helsinki regarding the Ethical Principles for Medical Research Involving Human Subjects. Informed consent: all patients participating in this study signed a written informed consent form for participating in this study. Patient consent for publication: written informed consent was obtained from a legally authorized representative(s) for anonymized patient information to be published in this article. Availability of data and materials: all data generated or analyzed during this study are included in this published article. Abstract The study aimed to validate the Manchester Triage System in a hospital setting using data for short- and medium-term death rates. A prospective observational study was conducted at the Emergency Department of Merano Hospital for two years. The discriminatory ability of MTS was tested using AUROCs and contingency tables, reporting sensitivity and specificity levels for each study outcome. A total of 98,443 patients were enrolled, 237 of whom died within 72h; 422 patients died within seven days, and 1025 died within 30 days. The MTS demonstrated excellent discriminatory ability, reporting AUROC values of 0.890 for death within 72h, 0.853 for death within seven days, and 0.781 for death within 30 days. A sensitivity of 87.7% and a specificity of 79.4% were reported for death at 72h, while a sensitivity of 69.6% and a specificity of 79.8% were reported for death at 30 days. The MTS has proven to be a good triage system capable of accurately identifying patients who are at risk of death in the short or medium term. Introduction The progressive increase in the number of requests for evaluation in hospitals’ Emergency Departments (EDs) after the COVID-19 pandemic, given an ever-decreasing number of available healthcare resources, makes triage one of the most important components of the emergency medical system.1,2 At present, internationally validated triage systems (e.g., the Manchester Triage System, the Emergency Severity Index, and the South African Triage Scale) that can be compared between different regions and nations remain rare. Although there are recommendations supporting the use of globally standardized triage systems, many European countries today have adopted autonomous systems created according to local needs. These systems often lack scientific validation, making it impossible to perform comparisons.3-6 In addition to being an expensive choice, given the significant amounts of resources required for the creation and design of new triage tools, these systems often merely simulate or integrate triage methodologies already widely developed in validated systems, hence becoming a pure relocation exercise that limits the development of a crucial aspect of the ED. A recent review of the literature on the performance of validated triage systems showed that there were no substantial differences in the ability to prioritize patients between the various systems and emphasized that regardless of the instrument used, performance remains comparable.3 The decision not to import one of these already structured systems to invest in local systems has led to the failure to evolve international triage systems, thus leaving them virtually unchanged since their creation in the 1990s.3-5 One of the most widespread and globally used systems is the Manchester Triage System (MTS), a system that classifies patients into five priority levels in association with 53 symptom-specific flow charts.7-9 General information on the ability of MTS to correctly stratify patients is limited, and few studies are validating the performance of the system against short- or medium-term mortality.4,8,10 Moreover, validation studies often used outcomes other than mortality and are often subject to the subjectivity of the evaluators (e.g., a pool of experts).4,8,10,11 Currently, there is no single triage system in Italy that is the standard for EDs in different regions; as a result, each department employs a contextual method with limited supporting scientific evidence. Moreover, the use of the MTS in Italian emergency and urgent care settings has not yet been examined. The present study aimed to validate the MTS against short-term (72 hours) and medium-term (7-30 days) mortality in an Italian context. Materials and Methods Setting The present single-center prospective observational study was conducted at the ED of the Merano General Hospital from 1 January 2021 to 31 December 2022. The ED under study has been using the MTS as a triage system since 2014. Triage is performed by two dedicated nurses during the day shift (08.00-20.00) and by one nurse during the night shift (20.00-08.00). The nurses performing triage in the ED must have at least two years of experience in a critical area setting; they must have completed a two-day course on the MTS method and must have undergone a six-month coaching period with an experienced triage nurse. Manchester Triage System The MTS is based on stratifying patients into five priority levels, where code 1 (red, immediate) stipulates a waiting time until medical attention of 0 minutes; code 2 (orange, urgency) defines a target waiting time of 10 minutes; code 3 (yellow, urgency) specifies a target waiting time of 60 minutes; code 4 (green, deferrable urgency) specifies a waiting time of 120 minutes, and code 5 (blue, non-urgent) specifies a waiting time of 240 minutes.11 In the case of a longer waiting time than that established by the priority code, the triage nurse must perform a new assessment of the patient to check their condition and consequently confirm or change the previously assigned priority code. The classification system of the MTS is based on 53 specific symptom diagrams (e.g., chest pain, palpitations, or wounds), where a different flow chart is associated with each symptom. Each flow chart has rapid indicators (specific symptom questions) to be scrolled through that define the level of priority from most urgent to least urgent. If no indicators are positive, the patient is classified with a priority code of 5. At the ED under study, the 3rd edition of MTS was used, using the German text.10 Patients and variables All patients admitted to the ED during the study period were enrolled. Only patients that were non- residents (e.g., tourists) of the province under study were excluded due to the impossibility of reconstructing the subsequent outcome. For all patients who consented to participate, their data, completed triage forms, and ED medical records were collected. All information gathered was subsequently entered anonymously into an electronic database. Outcome The primary outcome of the study is composed of sensitivity, specificity, positive predictive value, negative predictive value, and the area under the ROC curve (AUROC) of the triage assessment compared to the 72-hour mortality. The secondary outcomes are composed of sensitivity, specificity, positive predictive value, negative predictive value, and the area under the AUROC of the triage assessment compared to seven and 30 days mortality. The outcome was reconstructed by manual re-evaluation of medical records in the case of hospitalized patients or via the register office. Ethical considerations The study was approved by the local ethics committee (Comitato etico per la sperimentazione clinica, Azienda Sanitaria dell’Alto Adige, Bolzano, Italia, approval number 95-2019) and was conducted in accordance with the Declaration of Helsinki regarding the Ethical Principles for Medical Research Involving Human Subjects. Statistical analysis Continuous variables were described as the mean or interquartile range. The comparisons between different patient classes were performed with the Mann-Whitney test or the Kruskal-Wallis test as appropriate. Categorical variables were expressed as a percentage of the number of events out of the total, and comparisons between the patient classes were performed with Fisher's exact test or the Chi-square test as appropriate. The validity of the MTS was assessed by casting the data into 2×2 contingency tables to compare short-term (72 hours and seven days) and medium-term (30 days) mortality with MTS codes, where priority codes 1 (red) and 2 (orange) were combined as high priority codes, and priority codes 3 (yellow), 4 (green), and 5 (blue) was considered as low priority codes. This choice was made in agreement with previous studies. The 2×2 contingency tables were used to examine sensitivity (the ability of the MTS to correctly identify persons with high priority codes as being at risk of death in the short to medium term), specificity (the ability of the MTS to identify patients with low priority codes who did not die in the short to medium term), negative predictive value (NPV) (the probability of not dying in the short to medium term when the MTS assigned a low priority code), and positive predictive value (PPV) (the probability of dying in the short to medium term when the MTS assigned one of the high priority codes). The five priority levels of the MTS were also assessed via the AUROC. Subsequently, analyses were performed via the ROCs for subgroups of patients according to age (≥75 years, ≥65 years, >35 and <65 years, and ≤35 years), entry problem (medical-surgical entry symptoms, thus excluding fast-track), mode of arrival (out-of-hospital service or self-care), and conclusion after evaluation (admission or discharge). All results were described with their 95% confidence intervals (95% CI). Statistical analysis was performed using STATA 16.0 (StataCorp, College Station, TX, USA). Results The number of patients admitted between 2021 and 2022 was 120,258 (Figure 1). There were 98,443 patients enrolled in the study, of whom 0.4% (367/98,443) were code 1 (red), 4.7% (4636/98,388) were code 2 (orange), 5.6% (15,381/98,443) were code 3 (yellow), 69.5% (68,388/98,443) were code 4 (green) and 9.8% (6971/98,443) were code 5 (blue). The characteristics of the patients are shown in Table 1. Patients with higher codes were older and came to the ED more often by ambulance, by an emergency physician, or by helicopter. Patients with higher codes were more likely to be placed on a stretcher or in a wheelchair in the ED and to have more interventions performed in triage (ECG, blood sampling, or venous access placement). Patients assigned higher codes experienced more hospitalizations and more short- and medium-term deaths. In contrast, patients with lower codes were younger and were more likely to have reached the ED independently. In addition, patients with lower codes had fewer altered vital signs in triage and required fewer admissions. Overall, the percentages of patients who died at 72h, seven days, and 30 days were 0.2% (237/98,443), 0.4% (422/98,443), and 1.0% (1025/98,443), respectively. The median Length Of Stay (LOS) in the ED was 1.7 hours (95% CI: 0.9-2.9), while the median triage time was 3.3 minutes (95% CI: 1.8-6.6). The discriminatory ability of the MTS against 72-h, 7-day, and 30-day mortality is shown in Figure 2. The AUROC values varied from 0.781 (95% CI 0.765-0.798) for 30-day mortality to 0.853 (95% CI 0.831-0.875) for 7-day mortality and finally to 0.890 (95% CI 0.865-0.916) for 72-hour mortality, performance for each cutpoint of the ROC and each outcome is reported in the Supplementary Table 1. Considering only patients assigned to the areas of medicine or surgery, the discriminatory ability scores of the MTS for 72-h, 7-day, and 30-day mortality were 0.877 (CI 0.851-0.952), 0.837 (CI 0.814-0.860), and 0.770 (CI 0.753-0.787), respectively. The sensitivity of the MTS and thus the ability to accurately classify patients as high-priority codes regarding mid- and short-term mortality was very high, ranging from 87.7% for death at 72 hours to 69.6% for death within 30 days. The specificity of the MTS and thus the assignment of low-priority codes to patients who subsequently did not have the study outcome demonstrated excellent performance, with values ranging from 79.4% for death at 72 h to 79.8% for death at 30 days. Subsequently, subgroup AUROC values were calculated in comparisons of short- and medium-term mortality (Table 3). In the comparisons of the groups of older adult patients (≥ 65 years and ≥ 75 years) in all outcomes considered, the performance of the system was excellent, with AUROC values ranging from 0.673 to 0.865. The best performance was reported in patients aged < 35 years, with AUROC values ranging from 0.942 to 0.946, demonstrating nearly perfect discriminatory ability (Table 3). The worst performance was reported in the subgroup of patients hospitalized after ED evaluation, with AUROC values ranging from 0.673 to 0.779; this was still considered a good performance for this subgroup (Table 3). Discussion The current study using a large cohort of patients from an Italian ED represents the first national- level validation of the MTS by examining the performance of the system regarding short-term mortality. Italy currently has no objectively validated national triage system. Instead, several different systems have been created over the years according to local needs, and these are often lacking external or even internal validation, and very few have been published.12 In addition, there is considerable subjectivity in many of these systems, as the triage nurses do not always perform their assessment using objective criteria, but rather base their decisions on prior knowledge and experience, a concept that is largely outdated in the triage setting.10,12-14 This study demonstrated the remarkable ability of the MTS in stratifying patients accessing the ED, identifying those at risk of death in the short and medium term. Thus, the study suggests that given the difficulty of unifying triage systems in Italy, the MTS could be implemented in all national EDs given its worldwide dissemination and previously published validation studies.10,13,14 In addition to being an objective system with good predictive ability and prioritization capability on objective data such as short-term mortality, the MTS could allow broad comparisons across the EDs among Western countries. The present study has important consequences for clinical practice. First, the study yielded high AUROC values in the short- and medium-term mortality comparisons. A previous validation study of the MTS performed in German-speaking European areas by Graff et al. reported an AUROC of 0.613 for 30-day mortality, and the result was confirmed by subsequent research evaluating the performance of the MTS based on selected flow charts, with AUROC values ranging from 0.682 to 0.834 for in-hospital mortality.14,15 The differences in results between the present and previous studies may lie in the improvements made in EDs under investigation (such as, for example, daily triage auditing practices), demonstrating that continuous and consistent attention can lead to higher performance and better outcomes for patients.16 The second novelty of the study in view of the current international literature is the choice of objective outcomes (72-h, 7-day, and 30-day mortality) for validation. In previous research, the outcomes chosen were generally subjective measures such as confirmation of code appropriateness as determined by a pool of experts or the use of reference standards.13 Reference standards, however, are created by a panel of experts as study outcomes for validating triage systems, but at present, there are no data available regarding their actual effectiveness or usefulness.13,17 The choice to use outcomes such as mortality, in contrast to previous studies that selected objective and subjective outcomes simultaneously, is supported by the absence of gold standards for the evaluation of triage systems.17,18 Furthermore, a review by Kuriyama et al. concerning the validity of five-level triage systems suggested the use of outcomes different from reference standards or expert consensus, precisely because of the inherent subjectivity and the limited number of patients potentially re-evaluable by the expert pool.17 However, it is important to point out the weaknesses of the objective outcomes considered; in fact, short- and medium-term mortality is undoubtedly influenced by the choices made and the events following the triage assessment.17 Furthermore, as reported by Challen, triage is not meant to diagnose a disease or predict death, but rather is meant to characterize urgency.18 Objective outcomes such as death may not be appropriate for all patients (e.g., patients with severe pain as the result of a limb fracture will not be at risk of death but will need a higher code given the need for rapid pain management).18 Third, considering subgroups of patients categorized by age, the MTS has been shown to perform well in both younger and older patient groups. Due to the increasing older adult population accessing the ED and their inherent frailty, research in recent years has focused on evaluating this class of patients.19,20 Brouns et al. in a retrospective study of the MTS evaluated the performance of the triage system toward in- hospital mortality by dividing subjects into older patients (>64 years) and adult patients (18-64 years).21 The results demonstrated a worse performance of the MTS for older patients (AUROC 0.71; 95%CI 0.68 - 0.74) compared to adults (AUROC 0.79; 95%CI 0.72 - 0.85), although indicating a good overall ability of the triage system.21 The worse performance of the MTS in this group of patients may be related to the high number of comorbidities typical in older patients, a factor that the MTS does not consider and that may worsen or alter the manifestation in the short term, resulting in unexpected outcomes that are difficult to detect in triage. Given the overall aging demographic and the increase of multi-pathology patients accessing the ED, it is necessary to study and understand how to integrate specific assessments within a triage system.22,23 The system should consider the patient's innate comorbidity quota, thus allowing the MTS to optimally classify older patients. Despite this limitation, the good performance against mortality supports the use of the MTS and argues for its future evolution rather than its abandonment. The present study has several limitations. First, the single-center nature of the study limits the generalisation of the results. Second, the non-consideration of more subjective endpoints such as the assessment of code adequacy may have biased the results. Given the number of patients, it was impractical to manually re-evaluate all relevant records. Third, the correct application of the MTS was not assessed; rather, this was monitored daily through direct feedback within the ED under study, however, it is not possible to be certain about the correct application of MTS.16 Fourth, the version used for the implementation of MTS is based on the German book and not on the original one. Fifth, the outcome used is limiting for the complete evaluation of a triage system and therefore the results cannot completely represent the real functioning of MTS. Conclusions This is the first validation study of the MTS performed against short- and medium-term death. The system demonstrated excellent performance within an Italian hospital setting. The identification of patients at risk of death in the short and medium term was quite accurate, even for the most challenging groups such as older adult patients. If these results were subsequently confirmed by further studies, the use of a validated triage tool throughout the Italian country could be recommended. References 1) Zaboli A, Brigo F, Sibilio S, et al. The impact of COVID-19 pandemic on the urgency of patients admitted to the emergency department. Int Emerg Nurs 2022;65:101229. 2) Gilbert A, Brasseur E, Petit M, Donneau AF, D'Orio V, Ghuysen A. Advanced triage to redirect non-urgent Emergency Department visits to alternative care centers: the PERSEE algorithm. Acta Clin Belg 2022;77:571-8. 3) Hinson JS, Martinez DA, Cabral S, et al. Triage Performance in Emergency Medicine: A Systematic Review. Ann Emerg Med 2019;74:140-52. 4) Zachariasse JM, van der Hagen V, Seiger N, et al. Performance of triage systems in emergency care: a systematic review and meta-analysis. BMJ Open 2019;9:e026471. 5) Christ M, Grossmann F, Winter D, Bingisser R, Platz E. Modern triage in the emergency department. Dtsch Arztebl Int 2010;107:892-8. 6) Garrafa E, Levaggi R, Miniaci R, Paolillo C. When fear backfires: Emergency department accesses during the Covid-19 pandemic. Health Policy 2020;124:1333-9. 7) Brigo F, Zaboli A, Rinaldi F, et al. The Manchester Triage System's performance in clinical risk prioritisation of patients presenting with headache in emergency department: A retrospective observational study. J Clin Nurs 2022;31:2553-61. 8) Azeredo TR, Guedes HM, Rebelo de Almeida RA, et al. Efficacy of the Manchester Triage System: a systematic review. Int Emerg Nurs 2015;23:47-52. 9) Zaboli A, Ausserhofer D, Sibilio S, et al. Nurse triage accuracy in the evaluation of syncope according to European Society of Cardiology guidelines. Eur J Cardiovasc Nurs 2022;21:280-6. 10) Gräff I, Latzel B, Glien P, et al. Validity of the Manchester Triage System in emergency patients receiving life-saving intervention or acute medical treatment-A prospective observational study in the emergency department. J Eval Clin Pract 2019;25:398-403. 11) Parenti N, Reggiani ML, Iannone P, et al. A systematic review on the validity and reliability of an emergency department triage scale, the Manchester Triage System. Int J Nurs Stud 2014;51:1062-9. 12) Canzan F, Allegrini E, Mezzalira E, et al. Il processo decisionale dell’infermiere nell’attribuzione del codice di triage: uno studio qualitativa descrittivo [Nurse decision making in triage cose assignment: a qualitative descriptive study]. Assist Inferm Ric 2022;41:15-22. 13) Zachariasse JM, Seiger N, Rood PP, et al. Validity of the Manchester Triage System in emergency care: A prospective observational study. PLoS One 2017;12:e0170811. 14) Gräff I, Goldschmidt B, Glien P, et al. The German Version of the Manchester Triage System and its quality criteria--first assessment of validity and reliability. PLoS One 2014;9:e88995. 15) Brutschin V, Kogej M, Schacher S, et al. The presentational flow chart "unwell adult" of the Manchester Triage System-Curse or blessing? PLoS One 2021;16:e0252730. 16) Zaboli A, Sibilio S, Magnarelli G, et al. Daily triage audit can improve nurses' triage stratification: A pre-post study. J Adv Nurs 2023;79:605-15. 17) Kuriyama A, Urushidani S, Nakayama T. Five-level emergency triage systems: variation in assessment of validity. Emerg Med J 2017;34:703-10. 18) Challen K. How good is triage, and what is it good for? Emerg Med J 2017;34:702. 19) van Dam CS, Hoogendijk EO, Mooijaart SP, et al. A narrative review of frailty assessment in older patients at the emergency department. Eur J Emerg Med 2021;28:266-76. 20) Elliott A, Hull L, Conroy SP. Frailty identification in the emergency department-a systematic review focussing on feasibility. Age Ageing 2017;46:509-13. 21) Brouns SHA, Mignot-Evers L, Derkx F, et al. Performance of the Manchester triage system in older emergency department patients: a retrospective cohort study. BMC Emerg Med 2019;19:3. 22) Elliott A, Taub N, Banerjee J, et al. Does the Clinical Frailty Scale at Triage Predict Outcomes From Emergency Care for Older People? Ann Emerg Med 2021;77:620-7. 23) Salvi F, Morichi V, Grilli A, et al. Screening for frailty in elderly emergency department patients by using the Identification of Seniors At Risk (ISAR). J Nutr Health Aging 2012;16:313-8. Submitted: 02-05-2023 Accepted: 30-06-2023 Early access publication: 11-06-2023 Table 1. Anamnestic and clinical characteristics of patients enrolled in the study divided according to the priority code assigned in triage. Heart Rate* was collected in 38,683 Patients; Respiratory Rate was collected in 29,967 Patients; Oxygen Saturation* was collected in 39,783 Patients; Systolic blood pressure was collected in 33,361 Patients; Temperature* was collected in 65,859 Patients. Variables Blue & Green priority code Yellow priority code Orange & Red priority code p-value Patients, n (%) 78,059 (79.3) 15,381 (15.6) 5003 (5.1) Age, years, mean (SD) 55.8 (25.1) 50.5 (28.2) 58.1 (25.7) <0.001 Modality of arrival in ED, n (%) Autonomous Ambulance Emergency physician Helicopter 66,895 (85.7) 10,517 (13.5) 580 (0.7) 67 (0.1) 9,043 (58.8) 5,420 (35.2) 808 (5.3) 110 (0.7) 2,161 (43.2) 1,869 (37.4) 856 (17.1) 117 (2.3) <0.001 <0.001 <0.001 <0.001 After triage positioned, n (%) Walk Wheelchair Stretcher 66,818 (85.6) 9,367 (12.0) 1,873 (2.4) 7,736 (50.3) 4,691 (30.5) 2,953 (19.2) 1,276 (25.5) 1,085 (21.7) 2,641 (52.8) <0.001 <0.001 <0.001 Vital parameters Heart Rate*, median (IQR) Respiratory Rate*, median (IQR) Oxygen Saturation*, median (IQR) Systolic blood pressure*, mean (SD) 85 (74-96) 16 (14-17) 98 (97-99) 137.6 (23.4) 85 (74-100) 16 (15-18) 98 (96-99) 137.7 (25.4) 90 (75-110) 18 (16-25) 97 (93-98) 136.7 (29.6) <0.001 <0.001 <0.001 0.128 Temperature*, median (IQR) 36.2 (36.0-36.5) 36.3 (36.0-36.8) 36.8 (36.4-36.8) <0.001 Performances done in triage by the nurse, n (%) Venous access ECG Blood Sampling 6,731 (8.6) 5,398 (6.9) 6,700 (8.6) 3,984 (25.9) 2,676 (17.4) 3,793 (24.7) 1,978 (39.5) 1,589 (31.8) 1,887 (37.7) <0.001 <0.001 <0.001 Area of the declared symptoms, n (%) Internal medicine Surgery Orthopaedics Urological Otolaryngology & dentistry Ophthalmology Paediatrics Gynaecological Psychiatric Dermatology 17,140 (22.0) 13,853 (17.7) 22,736 (29.1) 1,977 (2.5) 6,588 (8.4) 3,851 (4.9) 6,519 (8.4) 3,000 (3.8) 442 (0.6) 1,953 (2.5) 5,133 (33.4) 2,876 (18.7) 2,794 (18.2) 891 (5.8) 187 (1.2) 424 (2.8) 1,520 (9.9) 986 (6.4) 525 (3.4) 45 (0.3) 3,277 (65.5) 437 (8.7) 130 (2.6) 73 (1.5) 5 (0.1) 32 (0.6) 216 (4.3) 787 (15.7) 45 (0.9) 1 (0.0) <0.001 <0.001 <0.001 <0.001 <0.001 <0.001 <0.001 <0.001 <0.001 <0.001 Hospitalised, n (%) 4,094 (5.2) 3,454 (22.5) 2,562 (51.2) <0.001 ED abandonment before medial evaluation, n (%) 2400 (3.1) 132 (0.9) 29 (0.6) <0.001 Death within 72h, n (%) 29 (0.04) 51 (0.3) 157 (3.1) <0.001 Death within 7 days, n (%) 76 (0.1) 105 (0.7) 241 (4.8) <0.001 Death within 30 days, n (%) 311 (0.4) 289 (1.9) 425 (8.5) <0.001 Table 2. 2x2 contingency tables divided according to the three study outcomes and the priority code assigned in triage, where low priority code considers patients with code 5 (blue) and code 4 (green) and high priority code considers patients with code 3 (yellow), code 2 (orange) and code 1 (red). Sensibility, specificity, PPV and NPV with respective 95% confidence intervals were calculated for each contingency table. Death within 72h No Yes Sensibility Specificity PPV NPV Low priority code 78.030 29 87.7% (82.9 – 91.6) 79.4% (79.2 – 79.7) 1.0% (0.9 – 1.1) 99.9% (99.9 – 100.0) High priority code 20.176 208 Death within 7 days No Yes Sensibility Specificity PPV NPV Low priority code 77.983 76 81.9% (81.7 – 82.2) 79.5% (79.3 – 79.8) 1.7% (1.6 – 1.8) 99.9% (99.8 – 99.9) High priority code 20.038 346 Death within 30 days No Yes Sensibility Specificity PPV NPV Low priority code 77.748 311 69.6% (69.4 – 69.9) 79.8% (79.5 – 80.1) 3.5% (3.4 – 3.6) 99.6% (99.6 – 99.6) High priority code 19.670 714 Table 3. AUROC of MTS performance calculated for subgroups of patients divided according to the three study outcomes. Death within 72 h Subgroups AUROC 95% Confidence Interval ≥ 75 years 0.863 0.835 – 0.892 ≥ 65 years 0.865 0.839 – 0.892 Between 35 & 65 years 0.890 0.866 – 0.913 <35 years 0.946 0.843 – 1.000 Internal Medicine & Surgery 0.876 0.852 – 0.900 Ambulance 0.802 0.762 – 0.841 Autonomous 0.865 0.806 – 0.924 Hospitalised 0.779 0.745 – 0.814 Discharged 0.863 0.810 – 0.917 Death within 7 days ≥ 75 years 0.809 0.784 – 0.834 ≥ 65 years 0.822 0.799 – 0.844 Between 35 & 65 years 0.853 0.832 – 0.873 <35 years 0.946 0.843 – 1.000 Internal Medicine & Surgery 0.836 0.815 – 0.858 Ambulance 0.758 0.727 – 0.789 Autonomous 0.796 0.744 – 0.849 Hospitalised 0.741 0.713 – 0.769 Discharged 0.792 0.745 – 0.840 Death within 30 days ≥ 75 years 0.742 0.724 – 0.761 ≥ 65 years 0.752 0.735 – 0.768 Between 35 & 65 years 0.781 0.766 – 0.796 <35 years 0.942 0.887 – 0.997 Internal Medicine & Surgery 0.770 0.753 – 0.786 Ambulance 0.685 0.664 – 0.706 Autonomous 0.730 0.699 – 0.760 Hospitalised 0.673 0.652 – 0.695 Discharged 0.702 0.675 – 0.729 Figure 1. Flow-chart of patients enrolled in the study. Figure 2: AUROC of MTS for the three selected outcomes. Supplementary Materials Table 1.Sensitivity, specificity and likelihood ratio values for each cut-point of the AUROC curves for the three study outcomes. Cut-point Sensibility Specificity LR+ LR- Death within 72 h ≥ Blue priority code 100.0% 0.0% 1.000 - ≥ Green priority code 99.1% 9.85% 1.099 0.085 ≥ Yellow priority code 87.7% 79.4% 4.271 0.154 ≥ Orange priority code 66.24% 95.1% 13.424 0.355 ≥ Red priority code 24.89% 99.7% 76.880 0.753 Death within 7 days ≥ Blue priority code 100.0% 0.0% 1.000 - ≥ Green priority code 98.6% 9.8% 1.093 0.144 ≥ Yellow priority code 81.9% 79.5% 4.011 0.226 ≥ Orange priority code 57.1% 95.1% 11.755 0.450 ≥ Red priority code 19.2% 99.7% 63.563 0.810 Death within 30 days ≥ Blue priority code 100.0% 0.0% 1.000 - ≥ Green priority code 97.8% 9.9% 1.086 0.216 ≥ Yellow priority code 69.6% 79.8% 3.449 0.380 ≥ Orange priority code 41.4 95.3% 8.823 0.614 ≥ Red priority code 11.7% 99.7% 44.378 0.8885