Journal Of Nursing Practice http://thejnp.org ISSN: 2614-3488 (print); 2614-3496 (online) Vol.5 No.1. October 2021. Page.126-135 126 A Comparison of Pain Scale on Neonates in a Top Refferal Hospital in Indonesia Nenty Septiana1, Yeni Rustina2, Defi Efendi2,3* 1Nursing Academy, Kesdam Iskandar Muda Banda Aceh, Banda Aceh, Indonesia 2Faculty of Nursing , Universitas Indonesia, Indonesia 3Neonatal Intensive Care Unit, Universitas Indonesia Hospital, Indonesia Corresponding author: defiefendi@ui.ac.id ABSTRACT Background: Pain assessment scale in neonates is the cornerstones of pain management so that the impact of pain can be prevented to maximize neonatal growth and development. Purpose: This study aimed to identify the most appropriate pain assessment scale used for neonates in Indonesia. Methods: A cross sectional study design was used in 30 neonates hospitalized using Neonatal Infant Pain Scale (NIPS), Neonatal infant Acute Pain Assessment Scale (NIAPAS) and Pain Assessment Tool (PAT). Pain assessment was performed by nurses (n=30) and expert nurses (n=5) participated in the validation of scales. Statistical analysis using validity (content, construct and concurrent validity) and reliability (inter-rater reliability and internal consistency) test. Results: NIPS instrument have excellent validity, reliability, and feasibility value compared with NIAPAS and PAT. Conclusion: NIPS was shown a valid, reliable, and practical scale for assessing pain in neonates. It allows nurses to identifying pain and help to provide of appropriate pain management. Keywords: Neonates, pain assessment, Neonatal Infant Pain Scale, Neonatal infant Acute Pain Assessment Scale, and Pain Assessment Tool. Received August 12, 2021; Revised August 21, 2021; Accepted September 17, 2021 DOI: https://doi.org/10.30994/jnp.v5i1.163 The Journal of Nursing Practice, its website, and the articles published there in are licensed under a Creative Commons Attribution- NonCommercial-ShareAlike 4.0 International License. http://thejnp.org/ mailto:defiefendi@ui.ac.id https://doi.org/10.30994/jnp.v5i1.163 Journal Of Nursing Practice http://thejnp.org ISSN: 2614-3488 (print); 2614-3496 (online) Vol.5 No.1. October 2021. Page.126-135 127 BACKGROUND Neonates are often exposed to painful procedures while being treated in the Neonatal Intensive Care Unit (NICU) (Witt et al., 2016), with an average of 10-16 painful procedures per day (Carbajal et al., 2008). Persistent pain stimulation at the beginning of life will result in physiological instability (Leena-Mari & Riikka, 2016), a high risk of intraventricular hemorrhage (Evans et al., 2005), damage to infant brain development, neurobehavioral development disorder (Cong et al., 2017), as well as contribute to future’s learning and behavior disorders (Badr et al., 2010). Pain management in neonates depends on early identification of the pain itself. Pain assessment is a challenge for nurses since neonates cannot verbally express the pain (Pölkki et al., 2014). However, neonates give pain cues through physiological, hormonal, and behavioral changes (Leena-Mari & Riikka, 2016). Pain assessment in neonates is crucial to identify the pain felt by using a screening tool (Cong et al., 2013). Pain assessment in neonates requires instruments that are valid, reliable, practical and must be multidimensional in observing the behavior and physiology shown by the neonate when feeling pain, it is to accurately determine the level of pain (O’Sullivan et al., 2016) so that proper pain management can be performed. Multidimensional pain screening tools consist of behavioral parameters (facial expressions, crying, body movements, muscle tone, skin color and sleep/wakefulness of the infant) and physiological parameters (heart rate variability, respiratory rate, blood pressure and oxygen saturation) (Holsti et al., 2011). Based on observations during practice in the perinatology room, nurses performed the screen pain for each baby treated using the Pain Assessment Tools (PAT) instrument. The results of interviews with five nurses perinatology found that the parameters assessed in the PAT instrument are still general, incomplete identifying pain in neonates, the score of each parameter is considered less representative of the parameters assessed, does not identify pain in neonates using intubation, and the difference between pain scores and no pain is two. This will affect the results of the infant's pain assessment. According to Hockenberry & Wilson (2015), the PAT instrument is used to assess postoperative pain not for procedural pain. Whereas what happens in the perinatology room, most neonates experience pain due to procedures such as peripheral blood sampling, heel prick, and suction (Mehrnoush et al., 2016). The validity and reliability of this instrument were not mentioned by the authors (Hockenberry & Wilson, 2015). Based on these data, researchers are motivated to explore appropriate pain instruments in assessing pain in neonates and can be applied in perinatology rooms. The author choose NIPS because multidimensional pain instrument that has adequate psychometrics because it has very good construct validity, concurrent validity, predictive validity, and inter-rater reliability which are important things in making an accurate assessment (Malarvizhi et al., 2012; Obiedat & Al-maaitah, 2020). In addition, NIPS is easy to identify procedural and postoperative pain in neonates without requiring special skills or tools in assessing pain and has been used worldwide (Pölkki et al., 2014). Meanwhile, NIAPAS was chosen because it is a new pain instrument developed by (Pölkki et al., 2014) with more specific and valid and reliable parameters. OBJECTIVE This study aimed to identify the most appropriate pain assessment scale used for neonates in Indonesia http://thejnp.org/ Journal Of Nursing Practice http://thejnp.org ISSN: 2614-3488 (print); 2614-3496 (online) Vol.5 No.1. October 2021. Page.126-135 128 METHODS Design dan Samples The design of this study was cross sectional. 30 neonates were selected by consecutive sampling according to inclusion criteria, namely 23-42 weeks' gestational age who were undergoing treatment and were going to undergo blood sampling procedures. Neonates who are analgesic or sedative and/or have a central nervous system (IVH grade 3 or 4) damage are exclusion criteria. The content validity of the NIPS, NIAPAS, and PAT instruments was assessed by five expert nurses (primary nurses, nurses at the level of ners education and nurses with work period > 5 years) by rating each physiological and behavioral indicators of the three pain instruments with a value of 1,2,3, and 4 (1 = not suitable, 2 = quite appropriate, 3 = suitable, 4 = very suitable). The accuracy of an instrument includes validity (construct and concurrent validity) and reliability (inter-rater reliability and internal consistency). Internal consistency was assessed by nurses (n=30) using video recordings on the NIAPAS, NIPS, and PAT instruments. Inter-rater reliability instruments were conducted to assess the suitability of the two observers to the pain assessment in neonates by two observers, which are expert nurses as perinatology primary nurses with work experience> 5 years and nurses at the level of ners education and both observers did not know the respondents who conducted the study (blind observation). Feasibility of the three instruments included the ease and duration of filling assessed by thirty nurses (n=30). Duration of filling instruments used the stopwatch. To investigate the ease of the instruments using open-ended question that are asked to nurses to measure how easily pain scale can be scored and interpreted, and how these instrumen can be used in clinical practice. Research Instrument and Data Collection Neonatal Infant Acute Pain Assessment Scale (NIAPAS): NIAPAS describes the level of pain before, during and after invasive procedures in neonates with a gestational age of 23- 42 weeks including 5 behavioral parameters (alertness, facial expressions, crying, body movements, reactions to touch); 3 physiological parameters (breathing pattern, oxygen saturation and heart rate); and gestational age. Each parameter has a score of 2, 3 or 4 and the total score of all parameters reflects the level of neonatal pain (Pölkki et al., 2014). Neonatal Infant Pain Scale (NIPS): NIPS can describe the level of neonatal pain of 28-40 weeks gestation carried out by invasive and postoperative procedures and is often used as a standard for screening pain in neonates, consisting of 6 parameters which are 5 behavioral parameters (facial expressions, crying, arm movements, limb movements, and awake status) and 1 physiological parameter (breathing pattern). Each parameter has a score of 0, 1, or 2 and the total score of all parameters can reflect the level of pain in neonates. A minimum score is 0 and a maximum score is 7. The higher the score, the higher the level of pain in neonates (Lawrence et al., 1993). Pain Assessment Tool (PAT): PAT is used to measure neonatal pain/discomfort of gestational age of 27-40 weeks undergoing surgery with the type of postoperative pain. PAT consists of 10 parameters which are 5 behavioral parameters (posture/tone, sleep patterns, facial expressions, crying, and skin color), 4 physiological parameters (respiration, heart rate, saturation, and blood pressure) and nurse's perception. Each parameter has a score range of 0-2. The minimum score is 0 and the maximum score is 20. The higher the score, the higher the level of pain in neonates (Hodgkinson et al., 1994). http://thejnp.org/ Journal Of Nursing Practice http://thejnp.org ISSN: 2614-3488 (print); 2614-3496 (online) Vol.5 No.1. October 2021. Page.126-135 129 Data Collection Pain assesstment is carried out in the Perinatologi room in referral hospital, Jakarta . Data collection stages included: 1) sample identifications according to inclusion and exclusion criteria and identifying neonatal characteristics; 2) The three instruments were translated into Indonesian by two translators. A week later, a back translation was carried out into English to assess the similarity of the words of the 3 instruments; 3) The researcher contacted the parents of neonates to explain the study procedure and informed consent. This study is voluntary and confidentiality is maintained. There are no extra painful procedures on neonates because of the study and clinical staff were advised to relieve pain of each neonates based on standard daily routine care in the unit. 4) Camera videos are positioned to get a clear picture of the neonates so that physiological parameters can be observed on a camera screen. Taking routine blood procedure on the heel or peripheral by clinical staff should be according to the standard hospital procedures. Only neonates participating in the study were recorded during the data collection period.; 5) recording one minute before, during, and 3 minutes after blood drawn; 6) neonatal pain assessment is carried out in three phases, one minute before, during, and three minutes after the blood draw procedure through video recordings which are performed 2-3 times on each research subject to confirm the neonatal response. The pain assessment for each neonate was assessed by one nurse using three pain instruments namely NIPS, NIAPAS, and PAT; 7) the researchers interviewed nurses who performed pain assessments related to the ease of these three instruments; 8) researchers calculate the duration of filling each pain instrument. Data Analysis: Univariate analysis was assessed to add up the characteristics of the respondents and the feasibility of the NIAPAS, NIPS, and PAT instruments, while the duration of the fill of the three instruments (minutes) used the mean value. The construct validity test uses the repeated ANOVA test in three phases of pain level measurement (before, during, and after blood collection), followed by the Bonferroni post hoc test to identify the differences between phases. The construct validity of an instrument is considered to be significantly good if the p-value <0.05. Each item statement of the three instruments is evaluated using a content validity index (I-CVI). Content validity is said to be very good if the mean I-CVI is ≥ 0.78 and the value of S-SVI / Ave ≥ is 0.90 (Polit & Beck, 2012). Concurrent validity was performed to see the correlation between items of statement of each instrument using the Pearson correlation test. Inter-rater reliability uses the Bland-Altmann test to assess the suitability of the two observers for the assessment of neonatal pain. Conformity is said to be good if it has a value of p> 0.05 (Dahlan, 2014). Internal consistency is assessed using the Cronbach's alpha test and is said to be good if the Cronbach's alpha value> 0.70 and very good if the value is in the range 0.80-0.9 (Scholtes et al., 2011);(DeVellis, 2017). http://thejnp.org/ Journal Of Nursing Practice http://thejnp.org ISSN: 2614-3488 (print); 2614-3496 (online) Vol.5 No.1. October 2021. Page.126-135 130 RESULTS The characteristics of respondents consisted of gender, birth weight, gestational age, form of breathing, length of stay and painful procedure. Tabel 1. Respondent’s demographic characteristic (n=30) Demographics f (%) Mean (SD) Range Gender Male 17 (56.7) Female 13 (43.3) Birth weight (g) 1934.4 935-4140 Less than 1000 4 (13.3%) 1000-1999 15 (50%) 2000-2999 6 (20%) 3000 or more 5 (16.7%) Gestasional age at birth (week) 33.6 (3.75) 27-39 Less than 28 weeks 2 (6.67) 28-31 weeks 8 (26.7) 32 – 36 weeks 10 (33.3) 37 weeks or more 10 (33.3) Form of breathing Need for mechanical ventilation 7 (23.3) Need for continous positive airway pressure 9 (30) Spontaneous breathing 14 (46.7) Length of stay (days) 15.13 (18.97) 1-69 Location blood draws Heel lance 8 (26.7%) Peripheral 22 (73.3%) Figure 1. Pain score before, during, and after blood draw procedure (n=30) 1.8 10.2 4.17 0.57 5.73 1.71.6 10.6 3.27 before procedure during procedure after 3 minute procedure Pain Neonates Score NIAPAS NIPS PAT http://thejnp.org/ Journal Of Nursing Practice http://thejnp.org ISSN: 2614-3488 (print); 2614-3496 (online) Vol.5 No.1. October 2021. Page.126-135 131 Table 2. Validity NIAPAS, NIPS, and PAT Instrument Instrument Validity Content (S-SVI/Ave-value) Validity Construct Validity Concurrent (r-value) Before (p-value) During (p-value) After (p-value) NIAPAS 0.75 p=0.000 p=0.000 p=0.002 0.391-0.825 NIPS 0.91 p=0.000 p=0,018 p=0.000 0.58-0.84 PAT 0.63 p=0.000 p=0.000 p=0.09 0.495-0.813 Table 3. Reliability, feasibility & clinical utility NIAPAS, NIPS, and PAT instrument Instrument Inter-rater Reliability Internal consistency (Cronbach’s alpha) Feasibility & Clinical Utility Before (p-value) During (p-value) After (p-value) Easy to administer and decode scoring of pain scale (%) Duration of filling in minutes (mean (SD)) NIAPAS 1 0.635 0.67 0.822 8 (25.7%) 6.82 (4.65) NIPS 0.814 0.13 0.74 19 (63.3%) 19 (63.3%) 3.85 (2.49) PAT 0.814 0.057 0.74 4.71 (2.27) 3 (10%) 4.71 (2.27) DISCUSSION The results showed the highest level of pain was during blood collection phase and the lowest was before blood collection phase. This is due to the painful stimulus, so that the infant responds to the stimulus through physiological and behavioral responses. Infants of gestational age <37 weeks are more sensitive to pain, have a low threshold of touch, more reflex responses to touch are seen, weakness of tone and muscle strength, and experience an increase in pain waves during blood sampling procedures compared to term infants (8,18,19). Exposure to recurrent pain and treatment that is not done causes changes in pain threshold, pain perception, and tolerance to pain throughout life. This happens because the pain pathway continues to develop during infancy and childhood. Besides, neonates will experience hyperalgesia (increased response to pain stimulus due to peripheral nerve sensitivity) and lower pain threshold will potentially lead to increased physiological and behavioural responses to painful events that may occur in the future (Ball et al., 2010) and will cause disruption in growth and the development of the infant's brain nervous system (Cong et al., 2017). Pain assessment in neonates requires a multidimensional, valid, reliable, and practical instrument for assessing pain levels (O’Sullivan et al., 2016). Instrument quality is determined by 3 domains, which are validity, reliability, and feasibility (Scholtes et al., 2011). Validity is a major component of an instrument (Tay & Jebb, 2017) and refers to the accuracy of measuring an instrument (Dharma, 2011) needed to show the credibility of a study (Sullivan, 2011). Validity consists of 3 components, such as construct validity, content validity, and concurrent validity (DeVellis, 2017). The results showed that pain level in each phase of the NIAPAS and NIPS instruments had decent construct validity compared to PAT. This can be seen from the significant increase in pain level in NIAPAS, NIPS, and PAT instruments from the phase before to http://thejnp.org/ Journal Of Nursing Practice http://thejnp.org ISSN: 2614-3488 (print); 2614-3496 (online) Vol.5 No.1. October 2021. Page.126-135 132 during the blood draw (p = 0.00), the phase before to after the blood draw (p = 0.002; p = 0.00; p = 0.091), and the phase during after the blood draw (p = 0.00, p = 0.018, p = 0.00). Of the three pain instruments, NIAPAS has the top construct validity value. The construct validity can evaluate the extent to which an instrument can measure what must be measured when compared with similar measurements it will interpret the same results (Scholtes et al., 2011). Furthermore, NIPS has the best content validity value compared to the others with the value of I-CVI≥0.78 for all statement items. Content validity is said to be very good if the average I-CVI≥0.78. The content validity of a measuring instrument is determined by its ability to measure the overall contents of the instrument to be measured (Polit & Beck, 2012). Concurrent validity was performed to see the correlation between items of each instrument statement using the Pearson correlation test (Scholtes et al., 2011). The results showed that the correlation between NIAPAS and NIPS in all three phases (before, during, and after) was in the range of 0.391 - 0.825. Then the correlation between PAT and NIPS is in the range 0.492-0.813. Lawrence et al. in his research showed that the correlation between NIPS and VAS in the phases before, during, and after blood collection was in the range of 0.58-0.84. This shows that NIPS has the best concurrent validity compared to NIAPAS and PAT (Lawrence et al., 1993). Besides validity, reliability is also fundamental in an instrument (DeVellis, 2017). Reliability is the level of consistency of measurement (Sullivan, 2011) including inter-rater reliability and internal consistency (Scholtes et al., 2011). The analysis showed that NIAPAS had the best match between observers, and was followed by NIPS. Furthermore, the highest Cronbach's alpha value was seen in the NIPS instrument compared to NIAPAS and PAT which was 0.896 so it was said that NIPS had the best internal consitency by showing a good correlation between statement items. The final results of the study indicate that NIPS is a valid and reliable pain instrument in assessing pain in neonates (premature and aterm). NIPS has the best content validity value, the best concurrent validity, good construct validity, proper match among the observers, and the best internal consistency compared to NIAPAS and PAT. According to Da Motta, Schardosim, & Da Cunha, pain instruments in neonates must be reliable, valid, and easy to use so that nurses could provide the optimal pain management easily (Da Motta et al., 2015). Multidimensional pain assessment is the best effort in assessing pain in neonates, it is by examining physiological and behavioural indicators. Physiological indicators can be seen from the saturation, frequency, and breathing pattern. On the other hand, behavioural indicators are seen from facial expressions, crying, body movements. Physiological and neonatal behavioural indicators are pivotal in assessing the level of pain in the neonate to determine the appropriate pain management in accordance with the level of neonatal pain (Cong et al., 2013). The American Academy of Paediatrics recommends that pain studies be carried out 4-6 hours routinely every day (Keels & Sethna, 2016), however, only 10% of neonates receive routine pain assessments while infants with mechanical ventilation are often not subjected to be assessed during treatment (Anand et al., 2016) NIPS has a sensitivity and specificity value of 85.94% (95% CI: 72.15-95.6%) and 92.61% which means NIPS has an educated ability to detect pain experienced by neonates and specifically identify painless neonates (Ge et al., 2015). Moreover, NIPS also has the shortest filling duration compared to others. NIAPAS has the longest filling duration of 6.82 minutes compared to NIPS and PAT. This is due to the difference between the calculation of the two parameters NIAPAS (oxygen saturation and pulse frequency) so it requires a lot http://thejnp.org/ Journal Of Nursing Practice http://thejnp.org ISSN: 2614-3488 (print); 2614-3496 (online) Vol.5 No.1. October 2021. Page.126-135 133 of time to calculate the difference in oxygen saturation/pulse frequency before and during blood draw. The ease of using the instrument is also largely determined by the nurse's perception of the instrument because nurses are key in assessing pain in neonates (Pölkki et al., 2014). The results of this study indicate that NIPS is the most easily filled and understood pain instrument compared to NIAPAS and PAT. Da Motta et al. explained that NIPS is a screening tool that is easy to understand and practical to use and requires a short time in detecting pain as well as describing NIPS as an efficient instrument for pain assessment in neonates (Da Motta et al., 2015). The results of applying the NIPS pain instrument can be applied because it is easy to understand, practical, and specific in assessing pain in neonates. This is in accordance with the conditions of nurses in a room that has a high workload. Mehrnoush et al., (2016) suggested that some barriers to effective pain management include workload, lack of personnel, lack of knowledge, lack of pain protocols, lack of time, lack of trust in pain assessment instruments. Strategies for improving the pain management including: providing education, using the latest journals, developing guidelines and providing support to nurses, developing pain assessment instruments that are clinically appropriate, providing adequate staff and proper workload, and the need for proper supervision and monitoring. The limitation of this study is that the use of video recording tool to assess pain scores makes it easier for different nurses to assess the pain scores. However, differences in the quality of the results of the recording will affect the level of accuracy of nurses in providing pain scores to infants compared to direct assessments. CONCLUSION This study concluded that NIPS has the best content validity, good construct validity, and very good concurrent validity, good inter-rater reliability value, excellent internal consistency, great sensitivity, and specificity value, has feasibility (ease, accuracy, and filling duration) that are better than NIAPAS and PAT instruments. Relevance to clinical practice The researcher recommends NIPS as a valid, reliable, easy, and practical instrument to be used in assessing pain in neonates. This could help nurses in assessing pain so that the pain management can be given appropriately and optimally. ACKNOWLEDMENTS We appreciate all the respondents and all the nurse Perinatology room of Dr. Cipto Mangunkusumo top referral Hospital Jakarta, Indonesia for contribution during the data collection. CONFLICT OF INTEREST We declare there is no conflict of interest in this study REFERENCES Anand, K. J. S., Eriksson, M., & Boyle, E. M. (2016). Assessment of Continuous Pain in Newborns admitted to NICUs in 18 European Countries study. International Journal of Laboratory Hematology, 38(1), 42–49. https://doi.org/10.1111/ijlh.12426 Badr, L. K., Abdallah, B., Hawari, M., Sidani, S., Kassar, M., Nakad, P., & Breidi, J. (2010). Determinants of premature infant pain responses to heel sticks. Pediatric Nursing, 36(3), 129–136. http://thejnp.org/ Journal Of Nursing Practice http://thejnp.org ISSN: 2614-3488 (print); 2614-3496 (online) Vol.5 No.1. October 2021. Page.126-135 134 Ball, J. W., Blinder, R. C., & Cowen, K. J. (2010). Children health nursing: Partnering with children & families (2nd ed.). Pearson Education. Bartocci, M., Bergqvist, L. L., Lagercrantz, H., & Anand, K. J. S. (2006). Pain activates cortical areas in the preterm newborn brain. Pain, 122(1–2), 109–117. https://doi.org/10.1016/j.pain.2006.01.015 Carbajal, R., Rousset, A., Danan, C., Coquery, S., Nolent, P., Ducrocq, S., Saizou, C., Lapillonne, A., Granier, M., Durand, P., Lenclen, R., Coursol, A., Hubert, P., De Saint Blanquat, L., Boëlle, P. Y., Annequin, D., Cimerman, P., Anand, K. J. S., & Bréart, G. (2008). Epidemiology and treatment of painful procedures in neonates in intensive care units. JAMA - Journal of the American Medical Association, 300(1), 60–70. https://doi.org/10.1001/jama.300.1.60 Cong, X., McGrath, J. M., Cusson, R. M., & Zhang, D. (2013). Pain assessment and measurement in neonates: An updated review. Advances in Neonatal Care, 13(6), 379– 395. https://doi.org/10.1097/ANC.0b013e3182a41452 Cong, X., Wu, J., Vittner, D., Xu, W., Hussain, N., Galvin, S., Fitzsimons, M., McGrath, J. M., & Henderson, W. A. (2017). The impact of cumulative pain/stress on neurobehavioral development of preterm infants in the NICU. Early Human Development, 108, 9–16. https://doi.org/10.1016/j.earlhumdev.2017.03.003 Da Motta, G. D. C., Schardosim, J. M., & Da Cunha, M. L. (2015). Neonatal infant pain scale: Cross-cultural adaptation and validation in Brazil. Journal of Pain and Symptom Management, 50(3), 394–401. https://doi.org/10.1016/j.jpainsymman.2015.03.019 Dahlan, S. (2014). Statistik untuk kedokteran dan kesehatan: Deskriptif, bivariat, dan multivariat dilengkapi dengan menggunakan SPSS seri 1 edisi 6. Sagung Seto. DeVellis, R. . (2017). Scale development: Theory and applications (ed.4th). Sage. Dharma, K. K. (2011). Metodologi penelitian keperawatan: Pedoman melaksanakan dan menerapkan hasil penelitian. Trans Info Media. Eckstein Grunau, R. (2013). Neonatal Pain in Very Preterm Infants: Long-Term Effects on Brain, Neurodevelopment and Pain Reactivity. Rambam Maimonides Medical Journal, 4(4), 1–13. https://doi.org/10.5041/rmmj.10132 Evans, J. C., McCartney, E. M., Lawhon, G., & Galloway, J. (2005). Longitudinal comparison of preterm pain responses to repeated heelsticks. Pediatric Nursing, 31(3), 216–221. Ge, X., Tao, J. R., Wang, J., Pan, S. M., & Wang, Y. W. (2015). Bayesian estimation on diagnostic performance of Face, Legs, Activity, Cry, and Consolability and Neonatal Infant Pain Scale for infant pain assessment in the absence of a gold standard. Paediatric Anaesthesia, 25(8), 834–839. https://doi.org/10.1111/pan.12664 Hockenberry, M. J., & Wilson, D. (2015). Wong’s essentials of pediatric nursing (9th. Ed.). Elsevier Mosby. Hodgkinson, K., Bear, M., Thorn, J., & Van Blaricum, S. (1994). Measuring pain in neonates: Evaluating an instrument and developing a common language. The Australian Journal of Advanced Nursing: A Quarterly Publication of the Royal Australian Nursing Federation, 12(1), 17–22. Holsti, L., Grunau, R. E., & Shany, E. (2011). Assessing pain in preterm infants in the neonatal intensive care unit: moving to a ‘brain-oriented’ approach. Pain Manag, 1(2), 171–179. https://doi.org/10.2217/pmt.10.19. Keels, E., & Sethna, N. (2016). Sedación Prevención y Manejo Dolor en Procedimientos. Pediatrics, 137(2), e20154271–e20154271. https://doi.org/10.1542/peds.2015-4271 Lawrence, J., Alcock, D., McGrath, P., Kay, J., MacMurray, S. B., & Dulberg, C. (1993). The development of a tool to assess neonatal pain. Neonatal Network, 12(6), 59–66. http://thejnp.org/ Journal Of Nursing Practice http://thejnp.org ISSN: 2614-3488 (print); 2614-3496 (online) Vol.5 No.1. October 2021. Page.126-135 135 Leena-Mari, H., & Riikka, L. (2016). Pain in premature infants : Pain reactions and possible consequences in later life. Thesis. https://www.theseus.fi/handle/10024/120228 Malarvizhi, G., Vatsa, M., Roseline, M., Nithin, S., & Paul, S. (2012). Inter-rater reliabilty of neonatal infant pain scale (NIPS) as a multidimensional behavioral pain tool. Nitte University Journal of Health Science, 2(1), 26–30. http://nitte.edu.in/journal/Marchsplit/Nitte University Journal March 2012_26_30.pdf%5Cnhttp://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference& D=emed10&NEWS=N&AN=2012600705 Mehrnoush, N., Ashktorab, T., Heidarzadeh, M., Momenzadeh, S., & Khalafi, J. (2016). Nursing care and documentation assistant with an electronic nursing management system in neonatal intensive care unit. Iranian Journal of Neonatology, 7(4), 23–29. https://doi.org/10.22038/ijn.2016.7854 O’Sullivan, A. T., Rowley, S., Ellis, S., Faasse, K., & Petrie, K. J. (2016). The Validity and Clinical Utility of the COVERS Scale and Pain Assessment Tool for Assessing Pain in Neonates Admitted to an Intensive Care Unit. Clinical Journal of Pain, 32(1), 51–57. https://doi.org/10.1097/AJP.0000000000000228 Obiedat, H., & Al-maaitah, E. I. (2020). Neonatal and Pediatric Medicine Critique of the Use of Neonatal Infant Pain Scale ( NIPS ). Neonatal and Pediatric Medicine, 6(1), 1– 5. Polit, D. F., & Beck, C. T. (2012). Nursing research: Generating and assesing evidence for nursing practice (9th ed.). Wolters Kluwer Health | Lippincot Wiliams & Wilkins. Pölkki, T., Korhonen, A., Axelin, A., Saarela, T., & Laukkala, H. (2014). Development and preliminary validation of the Neonatal Infant Acute Pain Assessment Scale (NIAPAS). International Journal of Nursing Studies, 51(12), 1585–1594. https://doi.org/10.1016/j.ijnurstu.2014.04.001 Scholtes, V. A., Terwee, C. B., & Poolman, R. W. (2011). What makes a measurement instrument valid and reliable? Injury, 42(3), 236–240. https://doi.org/10.1016/j.injury.2010.11.042 Sullivan, G. M. (2011). A Primer on the Validity of Assessment Instruments. Journal of Graduate Medical Education, 3(2), 119–120. https://doi.org/10.4300/jgme-d-11- 00075.1 Tay, L., & Jebb, A. T. (2017). Scale Development Scale Development. The SAGE Encyclopedia of Industrial and Organizational Psychology, January. https://doi.org/10.1007/978-3-8349-6891-3_3 Witt, N., Coynor, S., Edwards, C., & Bradshaw, H. (2016). A Guide to Pain Assessment and Management in the Neonate. Current Emergency and Hospital Medicine Reports, 4(1), 1–10. https://doi.org/10.1007/s40138-016-0089-y http://thejnp.org/