IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 306 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 DEVELOPMENT OF A RISK ASSESSMENT METHOD FOR SMALL-SIZED HOSPITALS USING AHP: A CASE IN NORTHERN INDIA Lakhwinder Pal Singh 1 Associate Professor, Department of Industrial and Production Engineering, Dr B R Ambedkar National Institute of Technology Jalandhar, Punjab (India)-144011 singhl@nitj.ac.in Harish Suthar Dr B R Ambedkar National Institute of Technology Jalandhar, Punjab (India)-144011 harishs.ip.16@nitj.ac.in ABSTRACT Several studies suggest that a major factor causing acute low back pain in nursing staff is the manual handling of disabled patients. In India, especially the northern part that consists of non-metropolitan cities, most of the hospitals still lack advanced patient handling methods and techniques. Therefore, it is necessary to devise a method of assessment for testing the effectiveness of an ergonomic intervention or training for safe patient handling. A proper quantification of risk involved in manual handling is required so that the severity of injuries caused by this handling can be reduced by an ergonomic intervention, which in turn helps redesign the task of manual handling. This study aimed to develop a qualitative method of risk assessment using the AHP for manual patient handling and to evaluate the validity and reliability of the risk assessment. The method is validated using the concepts of construct validity and content validity. The reliability was estimated through stability (test-retest) and homogeneity (internal consistency). The tests for validity and reliability were conducted in 130 units of 7 small-sized hospitals. The results of the current study reveal that the method was reliable and valid for risk assessment of patient handling. Keywords: Analytic Hierarchy Process (AHP); patient handling; risk assessment method for hospitals 1. Introduction The job of patient handling is physically demanding, and has a high risk for causing musculoskeletal disorders among caregivers (Putz-Anderson et al., 1997). Nurses are Acknowledgments: The authors would like to give their gratitude to the panel of experts, management and workers of the small-sized hospitals in the region, for their sincere cooperation in the collection of data. The authors are also thankful to staff of the government department for their co-operation and support. mailto:singhl@nitj.ac.in mailto:harishs.ip.16@nitj.ac.in IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 307 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 exposed to different risk factors including lifting and moving patients, pushing and pulling heavy equipment, working at forced postures or standing for long periods without taking adequate rest periods (National Research Council, 2001). The exceeding amount of work demands are present irrespective of the physical and mental capability of the workers, which also increases the risk of musculoskeletal injuries (Waters, 2010). A number of methods have been developed to evaluate posture during work that look at the handling of patients or any task. The Ovako Working Posture Analysing System (OWAS) (Karhu et al., 1981) is a simple and well-documented method. However, it does not differentiate between the right and left upper limbs; also, it does not evaluate the parts of the body such as the neck, elbows, and wrists. Additionally, the method does not consider repetition or duration of the sequential postures. On the other hand, another method, the Rapid Upper Limb Assessment (RULA), mainly focuses on disorders in the upper part of the body (McAtamney & Corlett, 1993), and provides a quick assessment of the upper body posture. The method for loading on the upper body (LUBA) evaluates the stress of working postures on the upper body (Kee & Karwowski, 2001). At the same time, the Rapid Entire Body Assessment (REBA) is applicable to the entire field where posture analysis is required. It provides a scoring system, which considers both static and dynamic activities (Hignett & McAtamney, 2004); however, it lacks detail and precision and only covers the force, repetition and posture risk factors. The duration and frequency of items are not considered, and the lack of time-based measures in REBA leads to the most common postures and the high duty cycle postures being ranked the same. 2. Literature review Researchers from different countries have reported different levels for the prevalence of back pain in one year; 47% in the United States (Trinkoff, 2002), 75% in Greece (Alexopoulos, 2003), 64% in Sweden (Johansson, 1995; Josephson, 1997), 66.8% in the Netherlands (Knibbe, 1996) and 68% in Switzerland (Maul, 2003). Some nurses suffering disabling back injuries have even had to abandon their jobs (Stubbs et al., 1986). Moreover, hospitals with higher incidence of such disorders among the workers have a higher staff turnover rate that further increases the costs of health care (OSHA, 2009). About 78% of the nurses who experienced back pain in the past 6 months did not report it to their management (Cato et al., 1989). Additionally, the work environment of public nurses has an increased risk for back pain in the lumbar region (Colombini et al., 1990). Many studies across the world have recognized the handling of patients as a high-risk activity and recommended a redesign of the practices (de Castro et al., 2006). Therefore, appropriate ergonomic intervention programs offer a great opportunity to reduce physical stress and the risk of injury in the lower backs of nurses (Garg et al., 1991). There are number of tools to assess risk in patient handling including the method of observation of risk, which describes and assesses the working technique used for the transfer of patients with respect to the safety and health of nurses. The method consists of 24 items grouped into three phases: preparatory phase, initial position and actual execution. After quantifying the valuations, the general score of the working technique with respect to the level of musculoskeletal risk and safety, the validity and reliability of the method was satisfactorily calculated (Johnsson, Kjellberg, Kjellberg, & Lagerstrom, 2004). Another method of observation contains 23 items and was tested with five different tasks of transferring patients, including a weighted score to assess IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 308 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 the mobilization technique used (Warming, Juul-Kristensen, et al., 2004). On the other hand, a system for comprehensive evaluation and a theoretical model to assess the risk of low back pain has also been used for analysis of the risk of low back pain and load on the spine during lifting of patients using different techniques (Marmaras et al., 1999b). However, despite all of these methods, currently, the MAPO (health care mobilization of patients hospitalized) is the most prominently used method. This method examines the analysis of risks in order to establish preventive measures, including the relocation of workers with limitations to the mobilization of patients (Battevi et al., 2006). To identify the major factors for making decisions or developing an evaluation method and their weights, there are a number of multiple criteria decision methods available. The Analytic Hierarchy Process (AHP), developed by Saaty (1990), is a combination of mathematics and interaction of the intended work (Viswanadhan, 2005; Wang & Wang, 2010). The AHP is one of the most successful techniques for solving decision making problems involving the goals, the alternatives for reaching the goals and the criteria for evaluating the alternatives (Harker & Vargas, 1987). A number of studies that applied the AHP include Andersson and Menckel, (1995), Arbel and Orgler, (1990), Armacost et.al. (1994), Badri (2001) and Bayazit (2005). Moreover, AHP has been successfully implemented in various organizations such as integrated manufacturing, layout design (Al-Harbi, 2001), assessment of technology asset decisions (Boucher & MacStravic, 1991), flexible industrialized systems and in many other engineering related fields (Arbel & Orgler, 1990; Armacost et al., 1994; Cambron & Evans, 1991; Das et al., 2012; Saaty, 1990; Shikdar and Al-Araimi, 2001). 2.1 Need for study The job of patient handling is physically demanding and has a high risk of musculoskeletal disorders among caregivers (Putz-Anderson et al., 1997). The exceeding amount work demands are present irrespective of the physical and mental capability of the workers and increase the risk of musculoskeletal injuries (Waters, 2010). Therefore, appropriate ergonomic intervention programs can offer a great opportunity to reduce physical stress and the risk of injury in the lower backs of nurses (Garg et al., 1991). Currently, there are several methods to evaluate risks, with different criteria that have a greater or lesser applicability depending on the working environment in which they are used. These methods are mainly developed and used for hospitals with advanced operational settings in developed countries. However, in developing countries like India, especially the northern state of Punjab with non-metropolitan cities like Jalandhar, Kapurthala, and Ludhiana, the majority of the hospitals are small-sized and lack advanced facilities, including adequate patient handling methods and techniques. Therefore, it is necessary to devise a method of assessment for testing the effectiveness of an ergonomic intervention or training for safe patient handling. In this study, an approach (AHP) based on the ability of mathematical structure of consistent matrices and the associated Eigenvectors to generate true or approximate weights was used. The AHP works on an Eigenvalue which is based on pair-wise comparisons (Bayazit, 2005; Boucher & Mac Stravic, 1991; Saaty, 1990). Qualitative and quantitative analyses can be performed simultaneously and calibration can be done using a suitable numeric scale (Saaty, 1985). In the current study, a method for assessing risk involved in patient handling is proposed for small-sized Indian hospitals. The current method is based on the method established in the past and includes some important customized and contemporary IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 309 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 parameters for the specific workplace setting. It also includes a validation and reliability analysis of the method by conducting observation in the case hospitals. 3. Methodology 3.1 Selection of aspects The methodology of the study mainly focuses on developing a method for the assessment of risk involved in patient handling activities in small-scale hospitals; therefore, to quantify the level of risk, the critical inherent aspects are considered. These aspects (items) are based on the three-observation method (Kjellberg et al., 2000; Johnsson et al., 2004; Villarroya et al., 2017). The eventual selection included eight items. i. Level of dependency (Weight= 3 points) ii. Climate conditions (Weight= 2 points) iii. Work place conditions (Weight= 5 points) iv. Primary aids (Weight= 6 points) v. Secondary aids (Weight= 6 points) vi. Work culture (Weight= 4 points) vii. Training (Weight= 2 points) viii. Risk perception (Weight= 2 points) Level of dependency of patients on the caregiver (J. Knibbe & Waaijer, 2005; Hignett, Fray, et al., 2014) has a maximum score of 3 points and depends on two factors including mobility level and cooperation of the patient. The mobility of patients is further divided into five levels (see Table 1) and cooperation level is divided into three levels as shown in Table 2. Table1 Mobility level and type of handling Sr. No Mobility level Type of handling 1 Level A: Moving patients and independent persons who dress and clean themselves. Safe handling: patients do not depend on the caregiver in any situation. 2 Level B: Patients able to stand and use a walker. Practically safe handling: patients depend on the caregiver in a few situations. 3 Level C: Patients who are partially supported, but require a wheelchair. Partially safe handling: patients depend on the caregiver in many situations. 4 Level D: Patients unable to stand on their legs. Practically unsafe handling: patients depend on the caregiver in most cases. 5 Level E: Patients who are completely immovable Unsafe handling: patients always depend on the caregiver. IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 310 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 Table 2 Cooperation level and type of handling Sr. No Cooperation Level Type of handling 1 Level 1: Fully cooperative Safe handling: When patients are cooperative (patients who cooperate with the caretaker during handling) 2 Level 2: Partially cooperative Partially safe handling: When the patients are partially cooperative (patients who have residual motor capacity) 3 Level 3: Non- cooperative Unsafe handling: When the patients are not cooperative (patients who cannot use the upper and lower parts of their body, and must be completely lifted in transfer operations). The score between the 15 combinations of mobility levels and levels of cooperation is distributed linearly in Figure 1. Figure 1 Linear distribution of score between various combinations The final score of the factor is obtained by multiplying the number of patients in each cell by the corresponding value, and dividing the sum of the number of patients analyzed as shown in Table 3. Table 3 Combined score of the different levels of mobility and cooperation Level (Cooperation V/S Mobility) Level A Level B Level C Level D Level F Level 1 3 2.7 2.4 2.1 1.8 Level 2 2.5 2.2 1.9 1.6 1.3 Level 3 2 1.7 1.4 1.1 0.8 IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 311 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 The climate conditions factor (weight = 2 points) checks whether the climate conditions are appropriate or not with scores given if the specific condition is fulfilled. Under the climate conditions, the following sub-factors are considered (see Table 4). Table 4 Point distribution for climate conditions Sr. No Sub-factor Condition Points 1 Temperature The temperature is between 14 and 27◦C. If fulfilled, 0.50 points; if not fulfilled, 0 points. 2 Humidity The relative humidity is 30% - 70%. If fulfilled, 0.50 points; if not fulfilled, 0 points. 3 Lighting 500 lux (equivalent to 70% of sun light) lightening. If fulfilled. 0.50 points; if not fulfilled, 0 points. 4 Noise The environment is less noisy If fulfilled, 0.50 points; if not fulfilled, 0 points. The workspace conditions factor (weight = 5 points) considers the condition of the workspace. It considers conditions such as access to bathrooms, condition of toilets, possibility of regulating beds for and the space of the rooms to safely perform patient handling. It has a maximum score of 5 points (see Table 5.). Table 5 Point distribution for workspace conditions Sub-factor Condition Point (i) Bathrooms Access to the bathroom without obstacles If fulfilled, 0.625 points If not fulfilled, 0 points Door width of at least 85 cm, and space for mechanical aids. If fulfilled, 0.625 points If not fulfilled, 0 points (ii) Water closet Commode is at least 50 cm high If fulfilled, 0.625 points If not fulfilled, 0 points Working space for handling a wheelchair If fulfilled, 0.625 points If not fulfilled, 0 points (iii) Adjustable beds Both the height and inclination can be changed If fulfilled, 1.25 points If not fulfilled 0 points (iv) Rooms Space between beds at least 90 cm If fulfilled 0.625 points If not fulfilled 0 points Free space of at least 120 cm between bed and wall If fulfilled 0.625 points If not fulfilled 0 points IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 312 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 The primary aids are those aids which directly help in handling and reduce the major portion of the load on the nurse. This factor has a maximum score of 6 points and considers the equipment available to perform the lifting or transfer of patients by means of primary aids (mobilization cranes, etc.). There are 1.5 points assigned for each type of existing aid that fulfills the prerequisites shown below and aids that were available in the unit in sufficient numbers. The primary aids include lifting equipment such as a handling crane and aids that help nurses remain in the correct posture such as wheelchairs, adjustable electric beds, adjustable stretchers etc. The prerequisites for the primary and secondary aids include being in the right state, appropriate training for use been given, easy to use for the nurses, and actually helps in handling. The secondary aids assist in handling and reduce the effort required from the nurse for handling. This factor has a maximum score of 6 points and includes access to the available equipment to perform the lifting, and transfer of patients, as well as the existence of minor aids (transfer, sliding sheets, etc.). A score of 1.5 points was assigned for each type of existing aid that fulfilled the prerequisites and the aids that were available in sufficient numbers in the unit. These aids should receive adequate maintenance so that they can be used for safe handling. The secondary aids include sliding sheets, transfer sheets, rotating discs, walkers and standing cranes. These secondary aids help nurses change the position of the patient and help in transportation and turning. The work culture factor has a maximum score of 4 points and considers the breaks given between work, ratio of patients per nurse, night work and support from co-workers in handling of patients (see Table 6). Table 6 Point distribution of work culture Sr. No Sub Factor Condition Score 1 Patient to nurse ratio An overall ratio of 10:1 If fulfilled, 1 point; if not fulfilled, 0 points. 2 Night shift No night shift If fulfilled, 1 point; if not fulfilled, 0 points. 3 Colleague Support If there is support from colleagues If fulfilled, 1 point; if not fulfilled, 0 points. 4 Rest No time pressures and rest is given between tasks If fulfilled, 1 point; if not fulfilled, 0 points. This training factor has a maximum score of 2 points and assesses the validity and effectiveness of the training given for patient handling (see Table 7). IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 313 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 Table 7 Point distribution of training Sr. No Sub-factor Condition Points 1 Information Information about the risks related to manual handling is given in training. If fulfilled, 0.5 points; if not fulfilled 0 points. 2 Theoretical training Theoretical and practical training is given to at least 75% of the workers. If fulfilled, 0.50 points; if not fulfilled 0 points. 3 Training period Training gap should not be more than 2 years. If fulfilled, 0.50 points; if not fulfilled, 0 points. 4 Evaluation of training Evaluation of training after 3 or 4 month of training. If fulfilled, 0.50 points; if not fulfilled, 0 points. This risk perception factor has a maximum score of 2 points. The perception of nurses was assessed by asking them some questions. The main objective was to determine the physical or mental load of patient handling. The details of the points assigned for risk perception are given in the Table 8. Table 8 Points assigned for risk perception Sr. No Sub factor Question Points i Danger Do you consider that the posture adopted during the handling of patients is dangerous to your health? If yes, 0.5 points; if no 0 points. ii Planning Is there any plan for the handling of the patients? If yes, 0.5 points; if no 0 points. iii Load Is patient handling a light activity? If yes, 0.5 points; if no 0 points. iv Rest Is there any need for rest in patient handling? If yes, 0.5 points; if no 0 points. 3.2 The AHP approach The AHP approach is based on the ability of the mathematical structure of consistent matrices and the associated Eigenvectors to generate true or approximate weights. The AHP works with an Eigenvalue which is based on pairwise comparisons (Bayazit, 2005; Boucher & Mac Stravic, 1991; Saaty, 1990). Qualitative and quantitative analyses can be performed simultaneously and calibration can be done using a suitable numeric scale (Saaty, 1985). The detailed procedure to carry out the AHP analysis consists of the following steps (Saaty, 1985; Saaty, 1990): IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 314 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 Step-1: Hierarchical structuring of a decision problem and selection of criteria. At the topmost level is the goal or focus. At the intermediate and lower levels are the criteria or sub-criteria and the available alternatives, respectively. Step-2: Construction of a pair-wise comparison matrix for each level with respect to the higher levels. In this step, the relative importance of different alternatives with respect to the immediately above sub-criteria is determined. This is followed by rating the relative priority of the criteria by assigning a weight between 1 (equal importance) and 9 (extreme importance) to the most important criterion. In contrast, the reciprocal of this value is assigned to the other criterion in the pair. Step-3: Application of Eigenvector methods to calculate the relative weight for the pairwise comparison of options on each criterion. Step-4: Check the consistency associated with the comparison matrix, using the consistency ratio (CR) of the consistency index (CI) with the appropriate value of the random index (RI). Step-5: Repeat the above steps for all levels in the hierarchy. Step-6: Evaluate the overall relative value by linear addition function. Each factor has a different weight based on their contribution to musculoskeletal disorders. The weight must be determined in terms of the relative importance of the item for scoring. The pairwise comparison, which is part of the AHP, is used for defining the weight of each factor. The weight of each factor shows the contribution of that factor in reducing risk exposure. For the pairwise comparison, two factors were evaluated at a time in terms of their relative importance. Index values from 1 to 9 and from 1 to 1/9 were used for the numerical rating. First, a comparison matrix was created. The weight of each factor comes as an Eigenvector also known as a priority matrix. Next, the Consistency Ratio (CR) was calculated to measure how consistent the judgments were relative to large samples of purely random judgments. If the CR is greater than 0.1 (10%), the judgments are untrustworthy because they are too close to randomness and the exercise is valueless or must be repeated. To find the Consistency Ratio (CR), first we must find the Consistency index (CI) with the help of the following equation: Consistency index (CI) = ( 𝜆𝑚𝑎𝑥− 1 𝑛 − 1 ) The Consistency Ratio (CR) was calculated using the method given by Saaty. In this case, there were 8 factors to be compared. Thus, the Consistency Ratio (CR) was calculated using the following expression: Consistency ratio (CR) = ( 𝐶𝐼 1.41 ) A consistency ratio of less than 0.1 implies that the pairwise comparison is consistent. IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 315 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 Consistency of pairwise comparison (𝐴𝑥 = 𝜆𝑚𝑎𝑥 𝑥) 𝜆𝑚𝑎𝑥 = 𝑀𝑎𝑥(10.73, 10.37, 9.32, 8.98, 8.98, 10.36, 10.42, 10.42) 𝜆𝑚𝑎𝑥 = 10.73 𝐶𝐼 = ( 10.73 − 8 8 − 1 ) = 0.14 𝐶𝐼 = 0.14 1.41 = 0.09 CR is less than 0.1, so the evaluations are consistent. The obtained weights are shown above in the list of the items. These items were further divided into sub-items and a relevant score was assigned to each sub-item. The study focused on the validity and reliability analysis of the method; therefore, the details of the score distribution were included in this paper. The score distribution is given in Figure 2. 3.3 Observation at hospitals The study included small-scale hospitals; the National Accreditation Board for Hospitals and Healthcare providers (NABH) defines a small-scale hospital as a hospital with less than 50 units. The data was collected from seven hospitals located in and around Jalandhar in Punjab (India). A total of 130 different units were observed during a period of 60 working days. These hospitals differ in their organizational structure and medical facilities (see Tables 9 and 12). The units were divided into three categories (I, II and III; see Table 12) depending on the type of wards or facilities. Since the cities under consideration are non-metropolitan in nature, the range of facilities available in each hospital is similar in kind and status. An agreement was made with the hospital management to not disclose their names, so an alphabetic-numeric code was assigned for each unit in the different hospitals.The sample was selected in the study based on non-probability convenience sampling. The willingness of the hospital management to participate in the survey study was a major consideration. The method proposed in the study also considered only those factors that are critical and for which the patient would not be disturbed during their acquisition. IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 316 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 Figure 2 Score distribution in factors (Level A. & Level 1.) - 3.0 points (Level A. & Level 2.) - 2.5 points (Level A. & Level 3.) - 2.0 points (Level B. & Level 1.) - 2.7 points (Level B. & Level 2.) - 2.2 points (Level B. & Level 3.) - 1.7 points (Level C. & Level 1.) - 2.4 points (Level C. & Level 2.) - 1.9 points (Level C. & Level 3.) - 1.4 points (Level D. & Level 1.) - 2.1 points (Level D. & Level 2.) - 1.6 points (Level D. & Level 3.) - 1.1 points (Level E. & Level 1.) - 1.8 points (Level E. & Level 2.) - 1.3 points (Level E. & Level 3.) - 0.8 points Temperature 0.5 or 0 points pintspoints Humidity 0.5 or 0 points Lighting 0.5 or 0 points points Noise 0.5 or 0 points Bathrooms 1.25 or 0 points Water closet 1.25 or 0 points Adjustable beds 1.25 or 0 points Rooms 1.25 or 0 points Handling Crane 1.5 or 0 points Wheelchair 1.5 or 0 points Adjustable electric bed 1.5 or 0 points Adjustable stretcher 1.5 or 0 points Dependency level Max score 3 points Climate condition Max score 2 points Workspace condition Max score 5 points Primary aid Max score 6 points Sliding sheet 1.5 or 0 points Transfer sheet 1.5 or 0 points Rotating disc 1.5 or 0 points Walker 1.5 or 0 points Patient to Nurse Ratio 1 or 0 points Night shift 1 or 0 points Colleague Support 1 or 0 points Rest 1 or 0 points Information 0.5 or 0 points Theoretical training 0.5 or 0 points Training period 0.5 or 0 points Evaluation of training 0.5 or 0 points Danger 0.5 or 0 points Planning 0.5 or 0 points Load 0.5 or 0 points Rest 0.5 or 0 points Secondary aids Max score 6 points Work culture Max score 4 points Training Max score 2 points Risk perception Max score 2 points Final score Max score 30 points IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 317 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 Table 9 Type and number of units in hospitals Hospital code Type No of units Hospital -H1 Government hospital 6 Hospital -H2 Psychiatric Hospital 9 Hospital -H3 Multi-specialist 25 Hospital -H4 Self-financing hospital 4 Hospital -H5 Multi-specialist 19 Hospital -H6 Heart specialist 25 Hospital -H7 Multi-specialist 42 Total Units observed 130 3.3.1 Risk levels obtained by the final score To obtain the final value of the risk level of the unit or service evaluated, the score obtained by all the items was added to a maximum of 30 points, which was subdivided into three risk levels (Table 10) with their respective color codes. Table 10 Categories for level of risk Risk level Score range Significance Green From 20.01-30 points. The risk of nurses suffering musculoskeletal disorders during the handling of the patient is acceptable. Yellow From 10.01-20 points. The risk of nurses suffering musculoskeletal disorders during the handling of the patient is moderate. Red From 0.8-10 points. The risk of nurses suffering musculoskeletal disorders during the handling of the patient is unacceptable. 3.4 Reliability of the method A method is considered highly reliable if it produces similar results under the same conditions. Various kinds of reliability coefficients, with values ranging between 0.00 (much error) and 1.00 (no error), can be used to indicate the amount of error in the scores. There are several general classes of reliability estimates; however, in the current study, there was a need to measure the test-retest reliability and internal consistency to test the reliability of the method. 3.4.1 Test-retest reliability There was only one observer to collect the data; therefore, consistency of measure was IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 318 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 assessed by calculating the test-retest reliability. There are various coefficients for measuring the test-retest reliability. For the data collected in the current study, the interclass correlation coefficient (ICC) is the appropriate measure for reliability because the quantitative measurements made on the units were organized into groups. Consistency was required between the different groups of the same type of unit in a hospital. The selection of the appropriate type of ICC was done by using the guidelines given by Koo and Li (2016). Two parameters need to be defined for a type of ICC, (a) model and (b) type. In this study, the rater (observer) was fixed and the subject was selected randomly, so the two-way mixed model was used (Model=3) with a single measure type (Type=1). The calculation of the ICC (3, 1) was done using the SPSS (Landers, 2015). Cicchetti (1994) gives the following (Table 11) guidelines for interpretation for ICC test-retest agreement measures: Table 11 Interpretation for ICC test measures 3.4.2 Internal consistency The classical measure for the internal consistency of an assessment method is Cronbach’s alpha (α). Cronbach’s alpha was calculated using the suitable statistical software package. 3.5 Validation of the method The construct validity is relevant in this study. Construct validity refers to the correctness of the conclusion about the relationship between two variables. In the current study, the data was collected using one method; therefore, it cannot be compared with the previous established method and accident rate in the hospital. Therefore, it is more important to consider the construct validity. 3.5.1 Construct validity The data was divided into three groups (Group I, Group II and Group III) to establish the construct validity (see Table 12). To establish the construct validity, three null hypotheses were proposed and tested for acceptance/rejection of the alternative hypotheses; therefore, a Student’s T-test was used to test the same at a 95% confidence level. Sr No. Agreement Level Measure 1 Less than 0.40 Poor 2 Between 0.40 and 0.59 Fair 3 Between 0.60 and 0.74 Good 4 Between 0.75 and 1.00 Excellent IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 319 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 Table 12 Groups of units and their description Groups Description Group I This group consists of the operation theaters, recovery units, psychiatric units, critical care units (CCU), surgical wards and intensive care units (ICU) because of the availability of the handling equipment in these units. Group II This group consists of all types of private rooms, which have more probability of risk exposure than Group I, but have less exposure then Group III. Group III This group includes all types of general wards. This group will have the lowest score which means higher risk exposure. 3.5.2 Content validity Content validity refers to how well the method measures what it intended to measure. In evaluation, the content validity index (CVI) is the most widely used index. Many studies suggest that factors with an I-CVI of 0.78 or higher for three or more experts could be considered evidence of good content validity. Also, a S-CVI (CVI) value greater than 0.9 shows good agreement between the experts. Therefore, in the current study the experts were from hospitals or were scholars who were already working in human safety. There were six experts who rated the factors considered in the method. 3.5.3 Calculation of most occurring factor in the observed hospital Simple arithmetic was used to find the most occurring factor that increased the risk exposure. The total score obtained for a factor was divided by the maximum score obtained; therefore, the conjugate of this fraction gave the contribution. Further, the most contributing factor can be found for each hospital and for all combined hospital units by using the following formula. % Occurance = (1 − Score obtained for a factor in a hospital maximum score which can be obtained ) × 100 4. Results To test the reliability of the assessment method, the interclass correlation coefficient (ICC) was calculated using SPSS; ICC values greater than 0.60 represent good test-retest reliability and a value less than 0.4 is considered poor reliability. The results of this phase may be observed in Table 13. IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 320 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 Table 13 Groups of units and their description Hospitals H1 and H4 were excluded from this test of reliability because the data from both hospitals were less as far as number of units to test reliability. The data showed two types of ICC, single measure and average measure. In this study, only one rater (observer) was present; therefore, the single measure ICC will give the correct measure of reliability. The results show the values of ICC (single measure) around 0.6, so the assessment is considered good if the reliability of the method is considered. Thus, the method tested in this section has good reliability. 4.1 Internal consistency The Cronbach’s Alpha (α) is a special type of ICC and is equivalent to the ICC (3, K). Therefore, to confirm the above results about the reliability of the method, it was obtained from SPSS as shown in Table 14. The value of alpha (α) should remain greater than 0.7 for consistency in the assessment. The values range from 0.7 to 0.8, which reflects the consistency of the tested method. Therefore, with the results of test-retest reliability and the internal consistency it can be concluded that the method used to assess risk exposure in the job of patient handling gives consistent results and is a stable procedure to assess the risk involved. Table 14 Result of internal consistency, the Cronbach’s Alpha (α) is a special type of ICC Hospital No. of Units Tested Cronbach’s Alpha (α ) H2 8 0.746 H3 18 0.756 H5 16 0.741 H6 14 0.716 H7 32 0.801 Hospital No of unit tested Interclass Correlational Coefficient (ICC) Single measure Average measure H2 8 0.595 0.746 H3 18 0.607 0.756 H5 16 0.588 0.741 H6 14 0.558 0.716 H7 32 0.668 0.801 IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 321 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 4.2 Validity 4.2.1 Construct validity The method is validated if Group (I) gets a higher score than Groups (II) & (III) and the score of Group (II) is greater than the score of Group (III). Therefore, the hypotheses assumed for equality of means has to be rejected with higher significance. The difference between the mean of the data of each group is shown in the Table 15. Table 15 Mean and variance of different group of units Group Mean N Std. Deviation Variance Group I 18.06 37 1.64 2.711 Group II 14.95 82 1.17 1.39 Group III 11.57 11 0.51 0.27 A mere observation of the data shown above confirms that the means of the data are decreasing continuously from Group (I) to Group (III). It was previously established that Group (I) should get a higher score because of the availability of handling equipment and other facilities in these units. This statement is supported by the higher mean of Group (I), also statistically confirmed for rejecting the hypothesis that the units receive an equal score. The t-test performed on the data of the three groups of units using SPSS gives the results exhibited in Table 16. Table 16 Results of construct validity Levene’s Test Equality of Means t-test for equality of Variances Hypothesis p value Mean difference t value Significance 1. Between Group I & II 0.03 3.11 15.55 a 0.001 a 2. Between Group II & III 0.009 3.38503 16.618 a 0.001 a 3. Between Group I & III 0.018 6.49509 20.767 a 0.001 a The SPSS package first performed a test for equality of variance known as Levene’s Test, where the value of ‘p’ is less than 0.05. The t-test performed on the condition of inequality of variance confirmed the statistically significant difference between the means of each group. Hence, the alternative hypothesis was accepted with a very high significance (95% level of significance). The value of ‘t’ was a high positive number, which means a large difference between the means of the scores. Group (I), Group (II) and Group (III) of the units and the positive value confirms that the score of Group I is higher than the score of Group II and so on. Therefore, the validation analysis of the method shows that this method is useful for assessing the risk exposure in patient handling as well as for finding the factor that requires immediate attention or is the main cause of inappropriate handling. Additionally, this method can be used to assess the effectiveness of the ergonomic intervention or the effect of employ new handling equipment. IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 322 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 4.2.1 Content validity Content validity refers to the extent to which the factors of a test or method can measure or assess what the test intended (Wynd et al., 2003). One of the most widely used measures of content validity is the content validity index (CVI). A panel of content experts was asked to rate each item on a scale in terms of its relevance to the underlying construct. The experts for this study included research supervisors and doctors from different hospitals. There were six experts used to determine the content validity. It is advisable to have a minimum of three experts to determine the CVI. The value of I-CVI should be greater than 0.75, and the value of S-CVI should be greater than 0.92 for the factor to be relevant for assessing the risk in patient handling. 4.2.2 Most occurring factor in observed hospital Figure 3 shows that factor 5 has the highest percentage occurrence (74.8%). Thus, based on the above results, it is concluded that factor 5 has the highest occurrence of contributing to an increased risk exposure among all the seven hospitals. Therefore, the inclusion of the secondary type of mechanical aids (such as handling crane, adjustable stretcher, adjustable electric bed, transfer sheets, standing crane, etc.) will reduce the risk exposure in the seven hospitals. Figure 3 Results of the most contributing factor when combining all hospitals IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 323 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 Description of the factors: Factor 1- Level of dependency Factor 2- Climate conditions Factor 3- Work places conditions Factor 4- Primary aids Factor 5- Secondary aids Factor 6- Work culture Factor 7- Training Factor 8- Risk perception The radar diagram exhibited in Figure 3 plots the overall contributing factors for all hospitals covered under this study. In six out of seven hospitals, factor 5 (secondary aids) emerges as the factor with the highest percentage contribution whereas only one hospital was found with factor 7 (training) as the most dominating factor. However, the overall radar diagram shows factor 5 as a most contributing factor confirming the results from previous diagrams. Figure 4 Percentage contribution of the each factor. Figure 4 above shows the highest percentage contribution (74.8%) for factor 5. Therefore, from all the above results, it can be concluded that factor 5 has the highest contribution to increasing the risk exposure in the sampled hospitals. The inclusion of the secondary type of mechanical aids (such as handling crane, adjustable stretcher, adjustable electric bed, transfer sheets, standing crane, etc.) will reduce the risk exposure in the hospitals. 5. Discussion 5.1 The observation method The observation method was developed to meet the need for a simple and practical tool to assess handling techniques during patient transfer tasks. The main application would be in evaluation of intervention effects. The method includes all required aspects of IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 324 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 patient handling, which various researchers previously used. These items are well tested in various parts of the world under different conditions. The new feature of the presented method is that it includes almost all the critical aspects of patient handling. The scoring system is based on documented risk factors for musculoskeletal disorders and on aspects of work technique related to generally accepted ergonomic principles. A similar system used in this method is called HEMPA (Kjellberg et.al, 2000). However, this method is partially different so that it is applicable in hospitals in non-metropolitan cities in developing countries like India. Here, some factors were excluded from the method to make it easier to implement. Therefore, it considers fewer factors than HEMPA. The excluded factors make the method more accessible for the nursing staff in the Indian health care industry and institutions. 5.2 Reliability of the method The reliability of the method was assessed using two similar aspects of reliability. The first was the test-retest reliability, which checks the agreement between observations taken on same subject at a different time points. The test-retest reliability was assessed using the interclass correlation coefficient (ICC). The value of ICC was approximately 0.6 for different hospitals, which reflects a good relationship between observations. The other aspect considered was internal consistency, which was assessed using the Cronbach’s alpha. Both the interclass correlation coefficient (ICC) and the Cronbach’s alpha showed similar results. The analysis of the internal consistency evaluation showed that the method presents homogeneity among its items. It was not necessary to remove any item from the instrument, as the data revealed an alpha ranging from 0.746 to 0.801 for different hospitals, demonstrating that the current proposed method is reliable. Hence, both the measures of reliability concluded that the method is reliable in its observation and produces consistent results. 4.3 Validity of the method Validity refers to the extent to which the method/instrument assesses the factors that it is supposed to. Validation of the current method was done by evaluating the construct validity. For the validation, the units were divided in three groups in terms of their facility for patient handling. The nursing staff working at Group II type of units was shown to be at greater risk than the nursing staff working at Group I units. The results of the method at Group I and Group II type of units confirmed the above observations. Therefore, the t-test was performed to statistically verify that Group I has a higher score than Group II, Group II has a higher score than Group III and eventually Group I has a higher score than Group III. The results of the t-test reject hypotheses with an equal score between the groups with a high significance level. Hence, the t-test confirmed that the score of Group I type of units is higher than Group II type of units, which eventually validates the method for assessing the risk exposure in patient handling. In addition, the content validity index shows that the factors chosen for the method are relevant to assessment of the risk exposure. 5.4 Applicability The method proposed and developed in this study is applicable in small-scale hospitals in non-metropolitan cities. This method is able to determine the prevalence of musculoskeletal disorder risk, low back injury, and other associated risks among the patient attendants. The individuals who have awareness and training with knowledge in transfer methods and ergonomics can easily adopt and use this method. There is a need to provide initial training to learn the items, definitions, and scoring system. The IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 325 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 method is applicable in wards, in patient’s homes or in other places where transfers occur, as no special equipment is needed. Therefore, it has significant applications in assessing risk exposure in patient handling. 6. Conclusion The handling of bedridden patients with mobility deficiencies constitutes one of the main risk factors for the health of their caregivers. In the health sector, there is a high incidence of musculoskeletal disorders. Many times, injuries are associated with the mobilization of patients, but also linked to the adoption of forced or static postures maintained over time, performing repetitive movements throughout the workday or manipulation of objects, such as medication carts or stretchers. Many epidemiological studies of personnel involved in the care of patients show the presence of acute low back pain with a higher prevalence with respect to other populations of workers exposed to physical risks. Therefore, the incorporation of mechanical equipment to help with mobilization is a remarkable preventive measure, since it promotes work postures and minimum manipulation effort by the worker. The current study developed a method to evaluate the risk exposure in patient handling, based on the previously established method. Construct validity presented a significant difference between all three groups of units, demonstrating that the method was able to detect these differences. The internal consistency analyzed by ICC and Cronbach’s Alpha coefficient demonstrated that the method is consistent in observation. The results indicated that the method is viable and may contribute to better planning for nursing assistance, together with other preventive ergonomic strategies in small-scale hospitals in non-metropolitan cities such as Jalandhar and areas around Punjab in India and in other developing countries. Therefore, the method is significant for health care workers to systematically plan the level of assistance needed in hospitals with settings and facilities similar to India. IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 326 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 REFERENCES Akarte, M., Surendra, N., Ravi, B., and Rangaraj, N. (2001). Web based casting supplier evaluation using analytical hierarchy process. Journal of the Operational Research Society, 52(5), 511-522. Doi: https://doi.org/10.1038/sj.jors.2601124 Al-Harbi, K. M. A. (2001). Application of the AHP in project management. International Journal of Project Management, 19(1), 19-27. Doi: https://doi.org/10.1016/s0263-7863(99)00038-1 Andersson, R., and Menckel, E. (1995). On the prevention of accidents and injuries: a comparative analysis of conceptual frameworks. Accident Analysis & Prevention, 27(6), 757-768. Doi: https://doi.org/10.1016/0001-4575(95)00031-3 Arbel, A., and Orgler, Y. E. (1990). An application of the AHP to bank strategic planning: The mergers and acquisitions process. European Journal of Operational Research, 48(1), 27-37. Doi: https://doi.org/10.1016/0377-2217(90)90058-j Armacost, R. L., Componation, P. J., Mullens, M. A., and Swart, W. W. (1994). An AHP framework for prioritizing customer requirements in QFD: an industrialized housing application. IIE Transactions, 26(4), 72-79. Doi: https://doi.org/10.1080/07408179408966620 Battevi, N., Menoni, O., Ricci, M. G., & Cairoli, S. (2006). MAPO index for risk assessment of patient manual handling in hospital wards: a validation study. Ergonomics, 49(7), 671-687. Doi: https://doi.org/10.1080/00140130600581041 Bordini, L., De Vito, G., Molteni, G., & Boccardi, S. (1999). Epidemiologia delle alterazioni muscolo-scheletriche da sovraccarico biomeccanico del rachide nella movimentazione manuale di pazienti. Medicina del Lavoro, 90(2), 103-116. Cato, C., Olson, D. K., & Studer, M. (1989). Incidence, prevalence, and variables associated with low back pain in staff nurses. AAOHN Journal, 37(8), 321-327. Doi: https://doi.org/10.1177/216507998903700804 Colombini, D., Riva, F., Lue, D., Nava, C., Petri, A., Basilico, S. & Menoni, O. (1999). Initial epidemiological data on the clinical effects in health workers employed in the manual lifting of patients in wards. La Medicina del Lavoro, 90(2), 201-228. De Castro, A. B., Hagan, P., & Nelson, A. (2006). Prioritizing safe patient handling: the American Nurses Association’s handle with care campaign. Journal of Nursing Adinistration, 36(7-8), 363-369. Doi: https://doi.org/10.1097/00005110-200607000-00009 Engkvist, I. L., Hagberg, M., Hjelm, E. W., Menckel, E., Ekenvall, L., & PROSA Study Group. (1998). The accident process preceding overexertion back injuries in https://doi.org/10.1080/00140130600581041 IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 327 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 nursing personnel. Scandinavian Journal of Work, Environment & Health, 367-375. Doi: https://doi.org/10.5271/sjweh.357 Garg, A., Owen, B., Beller, D., & Banaag, J. (1991). A biomechanical and ergonomic evaluation of patient transferring tasks: wheelchair to shower chair and shower chair to wheelchair. Ergonomics, 34(4), 407-419. Doi: https://doi.org/10.1080/00140139108967325 Goldman, R. H., Jarrard, M. R., Kim, R., Loomis, S., & Atkins, E. H. (2000). Prioritizing Back Injury Risk in hospital employees: Application and comparison of different injury rates. Journal of Occupational and Environmental Medicine, 42(6), 645-652. Doi: https://doi.org/10.1097/00043764-200006000-00016 Hignett, S., & McAtamney, L. (2004). Rapid entire body assessment. Handbook of human factors and ergonomics methods (pp. 97-108). CRC Press. Doi: https://doi.org/10.1201/9780203489925.ch8 Johnsson, C., Kjellberg, K., Kjellberg, A., Lagerstr¨om, M (2004). A direct observation instrument for assessment of nurses’patient transfer technique (DINO). In: Applied Ergonomics, 35(6), 591–601. Doi: https://doi.org/10.1016/j.apergo.2004.06.004 Karhu, O., Härkönen, R., Sorvali, P., & Vepsäläinen, P. (1981). Observing working postures in industry: Examples of OWAS application. Applied Ergonomics, 12(1), 13-17. Doi: https://doi.org/10.1016/0003-6870(81)90088-0 Kee, D., & Karwowski, W. (2001). LUBA: an assessment technique for postural loading on the upper body based on joint motion discomfort and maximum holding time. Applied Ergonomics, 32(4), 357-366. Doi: https://doi.org/10.1016/s0003-6870(01)00006-0 Kjellberg, K., Johnsson, C., Proper, K., Olsson, E., & Hagberg, M. (2000). An observation instrument for assessment of work technique in patient transfer tasks. Applied Ergonomics, 31(2), 139-150. Doi: https://doi.org/10.1016/s0003-6870(99)00046-0 Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155-163. Doi: https://doi.org/10.1016/j.jcm.2016.02.012 Landers, R. (2015). Computing Intraclass Correlations (ICC) as Estimates of Interrater Reliability in SPSS, The Winnower 2. Doi: https://doi.org/10.15200/winn.143518.81744 National Research Council (2001). Musculoskeletal disorders and the workplace: low back and upper extremities. Marmaras, N., Poulakakis, G., & Papakostopoulos, V. (1999). Ergonomic design in ancient Greece. Applied Ergonomics, 30(4), 361-368. Doi: https://doi.org/10.1016/s0003-6870(98)00050-7 IJAHP Article: Singh, Suthar/Development of a risk assessment method for small-sized hospitals using AHP: a case in northern India International Journal of the Analytic Hierarchy Process 328 Vol. 13 Issue 2 2021 ISSN 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 McAtamney, L., & Corlett, E. N. (1993). RULA: a survey method for the investigation of work-related upper limb disorders. Applied Ergonomics, 24(2), 91-99. Doi: https://doi.org/10.1016/0003-6870(93)90080-s OSHA (2009). Ergonomics for the Prevention of Musculoskeletal Disorders: Guidelines for Nursing Homes. Department of Labor, Occupational Safety and Health Administration. Doi: https://doi.org/10.1037/e372172004-001 Putz-Anderson, V., Bernard, B. P., Burt, S. E., Cole, L. L., Fairfield-Estill, C., Fine, L. J. & Nelson, N. (1997). Musculoskeletal disorders and workplace factors. National Institute for Occupational Safety and Health (NIOSH), 104. Saaty, T.L. (1990). How to make a decision: The Analytic Hierarchy Process. European Journal of Operational Research, 48, 9−26. Doi: https://doi.org/10.1016/0377-2217(90)90057-i Warming, S., Juul-Kristensen, B., et al. (2004). An observation instrument for the description and evaluation of patient transfer technique”. In: Applied Ergonomics 35.6, pp. 603–614. Doi: https://doi.org/10.1016/j.apergo.2004.06.007 Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin, 86(2), 420. Doi: https://doi.org/10.1037/0033-2909.86.2.420 Stubbs, D. A., Buckle, P. W., Hudson, M. P., Rivers, P. M., & Baty, D. (1986). Backing out: nurse wastage associated with back pain. International Journal of Nursing Studies, 23(4), 325-336. Doi: https://doi.org/10.1016/0020-7489(86)90055-6 Villarroya, A., Arezes, P., de Freijo, S. D., & Fraga, F. (2017). Validity and reliability of the HEMPA method for patient handling assessment. Applied Ergonomics, 65, 209-222. Doi: https://doi.org/10.1016/j.apergo.2017.06.018 Waters TR., (2010). Introduction to ergonomics for healthcare workers. Rehabilitation Nursing.