Archives of Academic Emergency Medicine. 2022; 10(1): e58 OR I G I N A L RE S E A RC H Evaluating the Ability of PRISM4 and PIM3 to Predict Mor- tality in Patients Admitted to Pediatric Intensive Care Unit; a Diagnostic Accuracy Study Victoria Chegini1, Hamidreza Hatamabadi2, Sima Jedari Attaran3, Abolfazl Mahyar3, Monirsadat Mirzadeh4, Venus Chegini5∗ 1. Department of Pediatrics, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran. 2. Department of Emergency Medicine, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran. 3. Department of Pediatrics, School of Medicine, Qazvin University of Medical Sciences, Qazvin, Iran. 4. Department of Community Medicine, School of Medicine, Qazvin University of Medical Sciences, Qazvin, Iran. 5. Department of Obstetrics and Gynecology, School of Medicine, Qazvin University of Medical Sciences, Qazvin, Iran. Received: May2022; Accepted: June 2022; Published online: 21 July 2022 Abstract: Introduction: Limited resources and the large number of children in need of services in the pediatric intensive care unit (PICU) emphasize the need for effective allocation of resources for improving the outcome of at-risk patients. This study aimed to evaluate and compare the accuracy of PRISM4 and PIM3 systems in prediction of in-hospital mortality of patients admitted to PICU. Methods: The present retrospective cross-sectional study was a diagnostic accuracy study performed on patients admitted to PICU of Qods Hospital, Qazvin, Iran, during one year. Scores of PRISM4 and PIM3 scales were calculated for each patient using the available calculators, and the outcome of patients regarding in-hospital mortality was recorded. Finally, screening performance charac- teristics of the mentioned scales in prediction of patients’ mortality were calculated and reported. Results: 218 patients with the mean age of 40.68 ± 37.92 (2-160) months were studied (57.8% female). There was a significant direct correlation between PIM3 score and duration of stay in PICU (p < 0.0001; r = 0.259), need for inotropic drug administration (p = 0.001), and mortality rate (p = 0.001). In addition, area under the receiver operating characteristic (ROC) curve of PIM3 and PRISM4 in prediction of mortality among patients admitted to the PICU was 0.939 (95%CI: 0.880 – 0.998) and 0.660 (95%CI: 0.371 – 0.950), respectively (p = 0.001). Based on the find- ings, the best cut-off point for PIM3 scale in prediction of mortality was the score of 4 and it was estimated to be the core of 8 for PRISM4 scale. Sensitivity and specificity of PIM3 scale in prediction of mortality in the cut-off of 4 points were 100.00 (95% CI: 56.09- 100.00) and 81.51 (95% CI: 75.47- 86.38), respectively. These measures were 42.85 (95%CI: 11.80- 79.76) and 98.10 (95%CI: 94.89- 99.39) for PRISM4 model, which indicates the higher sensitivity of PIM3 system in this regard. Conclusion: Based on the results of the present study, the accuracy of PIM3 is significantly higher than PRISM4 in prediction of in-hospital mortality among patients admitted to the PICU. It seems that considering the 100% sensitivity of PIM3 in prediction of outcome, this model is a better tool for screening patients who are at risk for in-hospital mortality in order to pay more attention and allocate more resources to improve their outcome. Keywords: Intensive Care Units, Pediatric; Mortality; Clinical Decision Rules; Prognosis Cite this article as: Chegini V, Hatamabadi H, Jedari Attaran S, Mahyar A, Mirzadeh M, Chegini V. Evaluating the Ability of PRISM4 and PIM3 to Predict Mortality in Patients Admitted to Pediatric Intensive Care Unit; a Diagnostic Accuracy Study. Arch Acad Emerg Med. 2022; 10(1): e58. https://doi.org/10.22037/aaem.v10i1.1738. ∗Corresponding Author: Venus Chegini; Department of Obstetrics and Gy- necology, School of Medicine, Qazvin University of Medical Sciences, Qazvin, Iran. Email: venus_chegini@yahoo.com, Phone: 00989123815963, ORCID: http://orcid.org/0000-0002-8031-5840 . This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem V. Chegini et al. 2 1. Introduction Among the most important aims in the pediatric intensive care unit (PICU) are providing quality care, providing relief for the disease without causing any permanent disabilities and side effects, and reducing the mortality of critically ill children. Since the foundation of the first PICU in 1955 in Sweden, considerable advances have been made in the qual- ity of care provided for the admitted patients, which has led to decrease in mortality rate from 8-18% in previous years to 3-5% (average 2.7%) in recent years (1, 2). This considerable achievement is the result of employing the best human re- sources and efficient use of the most recent diagnostic and therapeutic technologies. Evaluating the quality and efficacy of the measures and assessing the cost-effectiveness of inten- sive care is very important in efficient ressource allocation among patients (3). A reliable index of the efficiency of the mentioned unit is the standardized mortality ratio (SMR). SMR is the ratio of mor- talities observed in the studied group to the expected mortal- ity in the general public (2). Prediction of the risk of mortality and morbidity in patients admitted to the ICU is very effec- tive in improving the quality of measures taken and decreas- ing probable errors and can be of great help in allocating the available resources. In order to estimate the risk of mortality and the prognosis of the patient for allocating resources and evaluating the qual- ity of service, many scoring systems have been introduced to date, among which PRISM (pediatric risk of mortality) and PIM (pediatric index of mortality) are two of the major mod- els used for predicting mortality in the pediatric population (4, 5). PRISM uses the physiological and laboratory parameters in the first 24 hours of admission and the variables used in PIM include clinical and laboratory indices, underlying illnesses or surgical conditions leading to hospital admission. In the studies performed in various population, the results of evalu- ating the two models for determining which is superior in de- termining prognosis have been different (5). The study per- formed by EA Ozer et al. in 2004 on 105 babies less than 1- year-old in PICUs of hospitals in Turkey showed that PIM was a better scale than PRISM for predicting mortality in coun- tries with a higher rate of mortality and underlying disease (6). Meanwhile, in the study by Rajia et al. on 50 children in Egypt in 2019 showed that both systems had good predic- tions but PRISM was a better predictor (7). Therefore, the aim of this study was evaluating and compar- ing PRISM4 and PIM3 systems in predicting in-hospital mor- tality of patients admitted to PICU. 2. Methods 2.1. Study design and setting The present retrospective cross-sectional study is a diagnos- tic accuracy study performed on patients admitted to the PICU of Qods Hospital, Qazvin, Iran, from May 2019 un- til May 2020. The scores of PIM3 and PRISM4 scales were calculated for each patient using the available calculators and the outcome of patients regarding in-hospital mortal- ity was recorded. Finally, the screening performance char- acteristics of the mentioned scales in prediction of patient outcomes was calculated. The protocol of the study was approved by the ethics committee of Qazvin University of Medical Sciences (ethics code: IR.QUMS.REC.1399.182). The researchers adhered to the principles of ethics in biomed- ical research and observed confidentiality of patients’ data throughout the study. 2.2. Participants All the patients over 1 month and less than 12 years old, ad- mitted to PICU of Qods Children Hospital in Qazvin, Iran, were included in the study during the mentioned time using census method for sampling. Patients with the ICU stay of less than 12 hours or mortality within less than 12 hours were excluded from the study. In addition, profiles with missing or incomplete data regarding the required variables were also excluded from the study. It should be noted that the PICU of the hospital is managed with a closed system under the su- pervision of a fellowship of intensive care. 2.3. Data gathering Demographic and baseline characteristics of patients includ- ing age, sex, initial diagnosis, underlying illnesses, duration of hospitalization in the ICU, and all clinical and labora- tory variables required for calculation of PIM3 and PRISM4 scores, based on their latest versions (8, 9), were extracted from their clinical records and recorded in a designed check- list. In addition, outcomes such as need for mechanical ventilation, need for inotropic drug administration, duration of PICU stay, and in-hospital mortality after 12 hours were recorded for all patients. Then PIM3 and PRISM4 scores were calculated and recorded for all patients by a senior resident under the supervision of a pediatrician using online calcu- lators and entering the required variables based on the pa- tients’ condition upon admission. PIM and PRISM are two clinical decision rules or scoring systems that have been de- signed for predicting mortality among patients admitted to PICU and have changed during the recent years, the last ver- sions of which are available to researchers and practitioners under the names PRISM4 and PIM3. In a multi-center study published in 2019, which was performed on 10078 patients admitted to PICU, PRISM3 was revised and after undergo- This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem 3 Archives of Academic Emergency Medicine. 2022; 10(1): e58 Table 1: Characteristics of the studied population Variable Value Age (months) 40.68 ± 37.92 Gender Female 126 (57.8) Male 92 (42.2) Diagnosis Medical 199 (91.3) Surgical 19 (8.7) Underlying disease Immunodeficiency 3 (1.4) Malignancy 8 (3.7) Cardiovascular disease 16 (7.3) Hematologic disease 10 (4.6) Endocrine disorders 30 (13.8) Renal disease 10 (4.6) Musculoskeletal disease 29 (13.3) Vital signs Temperature (Celsius) 37 (36.6 – 37.7) Systolic blood pressure(mmHg) 107 (97 – 116) Diastolic blood pressure(mmHg) 64 (56 – 75) Heart rate (in a minute) 132 (116 – 154) Glasgow coma scale 15 (15 – 15 ) Outcomes Recovered and discharged 203 (93.7) Transferred to other hospitals 8 (3.7) Dead 7 (3.2) In need of mechanical ventilation 21 (9.6) In need of inotropic drugs 62 (28.4) Score PRISM4 1.6 (0.7 – 3.1) PIM3 1.0 (1.0 -1.0) Findings are presented as mean ± standard deviation, number (percentage) and median (interquartile range). ing changes for improving the accuracy of predictions and having less percent error it was reintroduced under the name PRISM4. 2.4. Outcomes The main outcome evaluated in the present study was in- hospital mortality of patients admitted to PICU. In addition, the duration of PICU stay and need for inotropic drugs were also studied as secondary outcomes. 2.5. Statistical analysis After filling out the checklists, the data were entered to SPSS software version 23 and analyzed. Data are reported as mean ± standard deviation (SD), frequency (%), or median (in- terquartile range (IQR)). Based on the distribution of samples in the population, correlation coefficient, chi square, t-test, or their non-parametric counterparts were used and, in all comparisons, level of significance was considered to be less than 0.05. To calculate the accuracy of the two mentioned systems in evaluation of patients’ outcomes, area under the receiver operating characteristic (ROC) curve was used and the screening performance characteristics of the two stud- ied scoring systems in the best cut-off points extracted from the ROC curve were calculated with 95% confidence intervals (CI) using VassarStats online calculator and reported. 3. Results 3.1. Baseline characteristics of the studied pa- tients 218 patients with the mean age of 40.68 ± 37.92 (range: 2-160) months were studied (57.8% female). Table 1 shows the base- line characteristics of the studied patients. The most com- mon chief complaints of patients on admission were respi- ratory problems (29.8%), and eating disorders (17.9%). The most common underlying illnesses among the studied cases were endocrine diseases (13.8%) and musculoskeletal dis- eases (13.3%). Mean duration of PICU stay was 5.94 ± 4.97 (Range: 1-45) days. Median and IQR of PRISM4 and PIM3 scores in the studied patients were 1.6 (0.7 – 3.1) and 1.0 (1.0 -1.0), respectively. The rate of mortality among patients ad- mitted to PICU in the present series was estimated to be 3.2% (7 cases). 3.2. Evaluating the predictive value of the stud- ied models There was a significant direct correlation between PIM3 score and duration of stay in PICU (p < 0.0001; r = 0.259), need for inotropic drug administration (p = 0.001), and mortality rate (p = 0.001). However, the correlation between PRISM4 score and duration of stay in PICU (p = 0.697; r = 0.27), need for inotropic drug administration (p = 0.139), and mortality rate (p = 0.107) was not significant. Area under the ROC curve of PIM3 and PRISM4 in predicting the need for inotropic drugs among patients admitted to PICU were 0.659 (95%CI: 0.575 – 0.743) and 0.558 (95%CI: 0.469 – 0.646), respectively (figure 1). Based on the findings, the best cut-off point for PIM3 scale in prediction of need for inotropic drugs was the score of 4 and for PRISM4 scale, it was estimated to be the score of 8. In addition, the area under the ROC curve of PIM3 and PRISM4 in prediction of the mortality of patients admitted to PICU were 0.939 (95%CI: 0.880 – 0.998) and 0.660 (95%CI: 0.371 – 0.950), respectively (figure 1). Based on these findings, the best cut-off point for PIM3 scale in prediction of mortality was the score of 4 and for PRISM4 scale, it was estimated to be the score of 8. Table 2 shows the screening performance characteristics of the two studied scales in prediction of mortality at their best cut-off points. Sensitivity and specificity of PIM3 model in This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem V. Chegini et al. 4 Figure 1: The area under the rock curve of PIM3 and PRISM4 criteria in predicting mortality (A) and the need for inotropic drugs (B) in patients admitted to the pediatric intensive care unit. Table 2: Screening performance characteristics of PIM3 and PRISM4 scoring systems in predicting the mortality of patients admitted in pedi- atric intensive care unit Character PIM3 PRISM4 TP 7 3 TN 172 207 FP 39 4 FN 0 4 Sensitivity 100.00 (56.09- 100.00) 42.85 (11.80- 79.76) Specificity 81.51 (75.47- 86.38) 98.10 (94.89- 99.39) Positive predictive value 15.21 (6.83- 29.48) 42.85 (11.80- 79.76) Negative predictive value 100.00 (97.27- 100.00) 98.10 (94.89- 99.39) Positive likelihood ratio 0.179 (0.089 – 0.358) 0.75 (0.25 – 2.18) Negative likelihood ratio 0 (0 – NaN) 0.019 (0.007 – 0.051) Data are presented with 95% confidence interval. NaN: the calculation cannot be performed because the entered values include one or more instances of zero. prediction of mortality at the cut-off point of 4 were 100.00 (95% CI: 56.09- 100.00) and 81.51 (95% CI: 75.47- 86.38), re- spectively. These measures were 42.85 (95%CI: 11.80- 79.76) and 98.10 (95%CI: 94.89- 99.39) for PRISM4, respectively, which indicates the higher sensitivity of PIM3 system for the mentioned purpose. 4. Discussion Based on the results of the present study, the frequency of in- hospital mortality in the studied PICU was 3.2% (7 cases out of the 218 cases admitted during one year). The accuracy of PIM3 system was significantly higher than PRISM4 system in the prediction of in-hospital mortality. It seems that PIM3 system is a better screening tool for pa- tients at risk of in-hospital mortality considering its 100% sensitivity. Unlike PRISM4, PIM3 system has a significant and direct correlation with duration of PICU stay as well as the need for inotropic drugs as patients with longer durations of PICU stay and those in need of inotropic drugs had higher PIM3 scores. In accordance with the present results, a study by Lopez (2018) demonstrated that PIM can adequately predict the mortality rate in ICU patients (10). Lee et al. in a study in 2017 showed that PIM3 score is a good predictor of the out- come for patients under 18 but not as efficient for hemato- oncologic patients (11). The performed study by EA Ozer et al. in 2004 on 105 babies less than 1-year-old in PICUs of hospitals in Turkey showed that PIM system is a better prognostic score than PRISM to predict mortality in countries with higher mortality rate and high prevalence of pre-existing medical conditions (6). A more recent study on 190 under 18 patients showed that PIM is a more accurate predictive score than PRISM score (12), in contrast to the study by Ragia et al. in Egypt which stated both of the systems were effective but PRISM was a better predictor (7). Another study performed on 145 PICU patients in India observed that PRISM3 score was associated with higher mortality rate and longer ICU stays, but it did not reliably estimate the survival rate, since sometimes patients with a very high PRISM3 score survived (11). Balkin morel et al. studied 14268 patients diagnosed with This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem 5 Archives of Academic Emergency Medicine. 2022; 10(1): e58 pulmonary hypertension in 153 PICUs in America and con- cluded that PIM2 and PRISM3 are both efficient scores for early identification and early treatment of high-risk patients and they can be used to update children’s parents on their treatment progress and outcome (13). Same results were ob- tained in Gandhi et al ‘s study in 2013. They proved that PIM2 can help predict the mortality rate in children and counsel their parents in the earliest stages of disease (14). Kadivar et al. declared that PRISM score measures the disease sever- ity and identifies the triage level of the children who derive greater benefits from PICU services (15). These results sup- port the results of Costa et al ‘s study in Brazil (16). According to the present study it appears that PIM3 system by a cut-off point of 4 is an accurate tool to help predict the likelihood of mortality and triage level of patients who are at lower risk. Our study suggests that sensitivity and specificity of this ver- sion of PIM enabled this scoring system as an effective clin- ical decision-making tool, because it can determine which patients actually need ICU admission and benefit the most from treatment in an intensive care unit. Meanwhile PRISM score had a greater specificity but very low sensitivity and this means PRISM score has a low positive predictive value for screening and predicting the outcome in the present study. On the other hand, the variables of PIM system require less laboratory testing, meaning It may be a faster and more effi- cient system to use. In our study PRISM system lacked the efficiency to predict the prognosis of PICU patients. This matter could be due to prevalence of pre-existing conditions and different types of medical conditions leading to hospital admission. There is therefore a definite need for multi-center studies with bigger sample sizes to settle this claim. Overall, clinical decision-making system is improving day by day to increase the quality of care and It seems that the fu- ture of clinical decision-making could be highly dependent on these scoring systems to enhance the patient’s outcomes. 5. Limitations These findings are limited by the use of retrospective de- sign because as we mentioned earlier profiles with missing or incomplete data regarding the required variables were ex- cluded from the study. Further studies with bigger sample sizes may help validate these results. 6. Conclusion Based on the findings of present study PIM3 is significantly more accurate in predicting the in-hospital death of PICU pa- tients than PRISM4 .It seems that considering the 100% sen- sitivity of PIM3 system in prediction of outcome, this model is a better screening tool for patients at risk of in-hospital mortality that actually need ICU admission and specialized care or need to be transported to a more equipped hospi- tal and also allocate more resources to them to prevent their death and improve their outcome. 7. Declarations 7.1. Acknowledgments We would like to express our appreciation to the ICU staff in Qods Hospital in Qazvin for their valuable effort during the conduction of the study. 7.2. Authors’ contributions ViC: devised the study, the main conceptual ideas and proof outline. HH and AM: Technical details, overall direction and planning SJA and VeC: Data gathering and calculating the scores MSM: Analysis and interpretation of the data All authors read, discussed the results, commented on, and approved the final version of manuscript. 7.3. Funding and supports This study is not funded. 7.4. Conflict of interest The authors declare no conflicts of interest in this study. References 1. Taori RN, Lahiri KR, Tullu MS. Performance of PRISM (Pediatric Risk of Mortality) score and PIM (Pediatric In- dex of Mortality) score in a tertiary care pediatric ICU. The Indian Journal of Pediatrics. 2010;77(3):267-71. 2. Qiu J, Lu X, Wang K, Zhu Y, Zuo C, Xiao Z. Comparison of the pediatric risk of mortality, pediatric index of mor- tality, and pediatric index of mortality 2 models in a pe- diatric intensive care unit in China: a validation study. Medicine. 2017;96(14). 3. Verlaat CW, Wubben N, Visser IH, Hazelzet JA, Van der Hoeven J, Lemson J, et al. Retrospective cohort study on factors associated with mortality in high-risk pediatric critical care patients in the Netherlands. BMC pediatrics. 2019;19(1):1-6. 4. Shann F, Pearson G, Slater A, Wilkinson K. Paediatric index of mortality (PIM): a mortality prediction model for children in intensive care. Intensive care medicine. 1997;23(2):201-7. 5. Pollack MM, Ruttimann UE, Getson PR. Pediatric risk of mortality (PRISM) score. Critical care medicine. 1988;16(11):1110-6. 6. Ozer EA, Kizilgunesler A, Sarioglu B, Halicioglu O, Sutcuoglu S, Yaprak I. The comparison of PRISM and PIM This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem V. Chegini et al. 6 scoring systems for mortality risk in infantile intensive care. Journal of tropical pediatrics. 2004;50(6):334-8. 7. Yousef RA, El Gendy FM, Abd El Aziz AA. Prognostic scor- ing systems in pediatric ICUs: pediatric risk of mortality III versus pediatric index of mortality 2. Alexandria Jour- nal of Pediatrics. 2019;32(1):27. 8. Pollack MM, Holubkov R, Funai T, Dean JM, Berger JT, Wessel DL, et al. The Pediatric Risk of Mortality Score: Update 2015*. Pediatric Critical Care Medicine. 2016;17(1):2-9. 9. Straney L, Clements A, Parslow RC, Pearson G, Shann F, Alexander J, et al. Paediatric index of mortality 3: an up- dated model for predicting mortality in pediatric inten- sive care. Pediatric critical care medicine. 2013;14(7):673- 81. 10. López MdPA, Boada N, Fernández A, Fernández AL, Ratto ME, Serrate AS, et al. Performance of the pediatric index of mortality 3 score in PICUs in Argentina: a prospec- tive, national multicenter study. Pediatric critical care medicine. 2018;19(12):e653. 11. Lee OJ, Jung M, Kim M, Yang H-K, Cho J. Validation of the pediatric index of mortality 3 in a single pediatric in- tensive care unit in Korea. Journal of Korean medical sci- ence. 2017;32(2):365-70. 12. Ramazani J, Hosseini M. Comparison of the predictive ability of the pediatric risk of mortality III, pediatric index of mortality3, and pediatric logistic organ dysfunction-2 in medical and surgical intensive care units. Journal of Comprehensive Pediatrics. 2019;10(2). 13. Balkin EM, Steurer MA, Delagnes EA, Zinter MS, Ra- jagopal S, Keller RL, et al. Multicenter mortality and morbidity associated with pulmonary hypertension in the pediatric intensive care unit. Pulmonary circulation. 2018;8(1):1-11. 14. Gandhi J, Sangareddi S, Varadarajan P, Suresh S. Pediatric index of mortality 2 score as an outcome predictor in pe- diatric Intensive Care Unit in India. Indian journal of crit- ical care medicine: peer-reviewed, official publication of Indian Society of Critical Care Medicine. 2013;17(5):288. 15. Kadivar M, Nourbakhsh S, Nouri K, SAMADI KP. Evalu- ating pediatric risk of mortality (PRISM) score in a pedi- atric critical setting: A prospective observational study in Children’s Medical Center. 2001. 16. Costa GA, Delgado AF, Ferraro A, Okay TS. Application of the pediatric risk of mortality (PRISM) score and deter- mination of mortality risk factors in a tertiary pediatric intensive care unit. Clinics. 2010;65:1087-92. This open-access article distributed under the terms of the Creative Commons Attribution NonCommercial 3.0 License (CC BY-NC 3.0). Downloaded from: http://journals.sbmu.ac.ir/aaem Introduction Methods Results Discussion Limitations Conclusion Declarations References