http://e-journal.unair.ac.id/JNERS | 33 Jurnal Ners Vol. 14, No. 1, April 2018 http://dx.doi.org/10.20473/jn.v13i1.10673 This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License Original Research Reliability and Validity Test of the Indonesian Version of the Hamilton Anxiety Rating Scale (HAM-A) to Measure Work-related Stress in Nursing Iwan Muhamad Ramdan Department of Occupational Health and Safety, Faculty of Public Health, Mulawarman University, Samarinda, Indonesia ABSTRACT Introduction: Specific work characteristics have placed nurses as one of the professions with a high level of work-related stress. If not managed properly, work-related stress can cause adverse effects. Signs of stress can be seen in people's behavior, thinking or physical symptoms. One of a subjective measurement tool that is widely used to measure work-related stress is the Hamilton Anxiety Rating Scale (HAM-A), however, the literature that discusses the results of the HAM-A translation, validity and reliability test in the nurse profession is still limited. This study aims to translate HAM-A into the Indonesian version, then test its validity and reliability in nurses. Methods: A Cross-sectional study with stratified random sampling method was conducted on 98 nurses from July to August 2018. The English version of HAM-A consists of 14 items has been a translation into Indonesian version. Pearson Product Moment Correlation was used to evaluate the construct validity and Cronbach's alpha scores were used to assess the internal consistency reliability of the Indonesian version of HAM-A. Results: Item construct validity based on the Pearson correlation ranged from 0.529 to 0.727, Cronbach’s alpha reliability was obtained at 0.756. Conclusion: The Indonesian version of the HAM-A fulfills the criteria of a reliable (fair acceptable criteria) and valid (good criteria) assessment tool to assess the work-related stress in the nursing profession. ARTICLE HISTORY Received: December 07, 2018 Accepted: August 30, 2019 KEYWORDS work-related stress; nurses; hamilton anxiety rating scale; reliability; validity CONTACT Iwan Muhamad Ramdan  iwanmuhamadramdan@gmail.com  Department of Occupational Health and Safety, Faculty of Public Health, Mulawarman University, Samarinda, Indonesia Cite this as: Ramdan, I. M. (2019). Reliability and Validity Test of the Indonesian Version of the Hamilton Anxiety Rating Scale (HAM-A) to Measure Work-related Stress in Nursing. Jurnal Ners, 14(1), 33-40. doi:http://dx.doi.org/10.20473/jn.v14i1.10673 INTRODUCTION Work-related stress is the response people may have when presented with work demands and pressures that are not matched to their knowledge and abilities and which challenge their ability to cope (World Health Organization, 2007)(International Labour Organization, 2016). Previous research concluded that work-related stress is associated and increase an individual’s vulnerability to burnout, job satisfaction and physical as well as mental health outcomes (Piko, 2006; Pillay, 2009). Other health problems related to the effects of work stress include: cardiovascular disorders (Li, Loerbroks, Bosma, & Angerer, 2016), gastrointestinal disorders (Huerta-Franco, 2013), musculoskeletal disorders (Nafeesa, Vidhya, Vijayalakshmi, & Rajkumar, 2017), anxiety and depression (Fan, Blumenthal, Watkins, & Sherwood, 2015), work fatigue (Rose et al., 2017), insomnia (Deguchi et al., 2017), alcohol abuse (Moore, Sikora, Grunberg, & Greenberg, 2007), decrease marital quality (Obradović & Čudina-Obradović, 2013) and disruption of social interaction (La Torre et al., 2018). Specific work characteristics such as working time, length of interaction with patients, emotional nature of patient demands and inter-professional relationships are prone to conflict (Khamisa, Peltzer, Ilic, & Oldenburg, 2017). Sources of work-related stress of nursing consist of working environment (physical, psychological and social environment factors), interpersonal relationships, nature of nursing, organizational factors, role characteristics and individual characteristics (Moustaka & Constantinidis, 2010). Work-related stress if not https://creativecommons.org/licenses/by/4.0/ https://creativecommons.org/licenses/by/4.0/ mailto:iwanmuhamadramdan@gmail.com I. M. RAMDAN 34 | pISSN: 1858-3598  eISSN: 2502-5791 managed properly can cause adverse effects, including emotional disturbances, behavioral problems, biochemical and neurohormonal changes, presenting added risks of mental or physical illness. Conversely, well managed work-related stress will create a feeling of mastery and self-confidence; increases motivation, working capacity and satisfaction; and improves health (Vernekar & Shah, 2018). Signs of stress can be seen in people's behavior changes. Acute responses to stress may be in the areas of feelings (for example, anxiety, depression, irritability, fatigue), behavior (for example, being withdrawn, aggressive, tearful, unmotivated), thinking (for example, difficulties of concentration and problem solving) or physical symptoms (for example, palpitations, nausea, headaches). If stress persists, there are changes in neuroendocrine, cardiovascular, autonomic and immunological functioning, leading to mental and physical ill health (for example anxiety, depression, heart disease) (Michie, 2002). Anxiety symptoms are serious and critical problems in the occupational context and they can be associated with stress (Vignoli, Muschalla, & Mariani, 2017). One subjective measurement tool that is often used to measure work-related stress is the Hamilton Anxiety Rating Scale (HAM-A) (Thompson, 2015). HAM-A was one of the first rating scales developed to measure the severity of anxiety symptoms, being considered one of the most popular assessment instruments widely used rating scales both clinical and research settings/general health psychology, has been widely translated into various languages in the world and is widely used to measure work stress in various types of work (López-Pina, Sánhez-Meca, & Rosa-Alcázar, 2009; Thompson, 2015). Research using HAM-A to assess work-related stress among others: Karanikola et al (2016) who examined anxiety symptoms and quality of interaction among Greek oncology nurses, and Craiovan (2015) which examines burnout, depression, and quality of life among the Romanian employees working in non- governmental organizations. The HAM-A has been translated into Cantonese for China, French, Urdu, and Spanish, however, the literature that discusses the results of the HAM-A translation, validity and reliability test in the nurse profession in the Indonesian version is still limited. To make the HAM-A accessible for the use in an Indonesian setting, especially in the nursing profession, this study aims to translate HAM-A into the Indonesian version, then test its validity and reliability among nurses in Indonesia. MATERIALS AND METHODS Design of Study and Participants A Cross-sectional study was conducted among 98 nurses at one of the government hospitals in East Kalimantan from July to August 2018. The research sample was taken stratified randomly from all departments (operation room, hemodialysis room, emergency room, internist room, pulmonary room, surgery care room, medical checkup, and polyclinics). Determination of respondents using random sampling method (Singh & Masuku, 2014). All members of the nurse population are given a serial number, the serial number is written on small paper and rolled up, then inserted into a closed glass which is given a small hole, the researcher then shakes the glass, each number that comes out is made a respondent. Instruments The English version of Hamilton Anxiety Rating Scale (HAM-A) consists of 14 items, each defined by a series of symptoms, and measures both psychic anxiety (mental agitation and psychological distress) and somatic anxiety (physical complaints related to anxiety) (Hamilton, 1959; Maier, Buller, Philip, & Heuser, 1988). The dimension of HAM-A consist of: (1) Anxious mood: Worries, anticipation of the worst, fearful anticipation, irritability; (2) Tension: Feelings of tension, fatigability, startle response, moved to tears easily, trembling, feelings of restlessness, inability to relax; (3) Fears: Of dark, of strangers, of being left alone, of animals, of traffic, of crowds; (4) Insomnia: Difficulty in falling asleep, broken sleep, unsatisfying sleep and fatigue on waking, dreams, nightmares, night terrors; (5) Intellectual: Difficulty in concentration, poor memory; (6) Depressed mood: Loss of interest, lack of pleasure in hobbies, depression, early waking, diurnal swing; (7) Somatic (muscular): Pains and aches, twitching, stiffness, myoclonic jerks, grinding of teeth, unsteady voice, increased muscular tone; (8) Somatic (sensory): Tinnitus, blurring of vision, hot and cold flushes, feelings of weakness, pricking sensation; (9) Cardiovascular symptoms: Tachycardia, palpitations, pain in chest, throbbing of vessels, fainting feelings, missing beat; (10) Respiratory symptoms: Pressure or constriction in chest, choking feelings, sighing, dyspnea; (11) Gastrointestinal symptoms: Difficulty in swallowing, wind abdominal pain, burning sensations, abdominal fullness, nausea, vomiting, borborygmi, looseness of bowels, loss of weight, constipation; (12) Genitourinary symptoms: Frequency of micturition, urgency of micturition, amenorrhea, menorrhagia, development of frigidity, premature ejaculation, loss of libido, impotence; (13) Autonomic symptoms: Dry mouth, flushing, pallor, tendency to sweat, giddiness, tension headache, raising of hair; (14) Behavior: Fidgeting, restlessness or pacing, tremor of hands, furrowed brow, strained face, sighing or rapid respiration, facial pallor, swallowing, etc. Each item is scored on a scale of 0 (not present) to 4 (severe), with a total score range of 0–56, where <17 indicates mild severity, 18–24 mild to moderate severity and 25–30 moderate to severe(Maier et al., 1988). HAM-A is comprised of a psychic and a somatic subscale. The psychic subscale (items 1-6 and 14) JURNAL NERS http://e-journal.unair.ac.id/JNERS | 35 addresses the more subjective cognitive and affective complaints of anxiety (e.g., anxious mood, tension, fears, difficulty concentrating), and is particularly useful in assessing the severity of general anxiety disorders (GAD). The somatic component (items 7- 13) emphasizes the features of GAD such as autonomic arousal, respiratory, gastrointestinal and cardiovascular symptoms (Katherine Shear et al., 2001). Cross-cultural Process of Daptation and Translation The questionnaire was translated according to the guidelines for cross-cultural adaptation of self-report measures by Beaton et al (2000). The guidelines consists of five stages: (1) Translation by two bilingual translators (native Indonesian), then they discuss and identify the selection of appropriate words so that they can reflect clinical symptoms and the language used by the general population; (2) Synthesis, both translators and an observer discuss to get a combined translation; (3) back translation by two bilingual translators (native English), the results of a combined translation in Indonesian are translated back into English by two translators with the aim of checking the validity of the translation process and ensuring the translated version reflects the same meaning as the original version; (4) evaluation by a team of experts consisting of forward and backward translators, epidemiologists and health professional experts, this was intended to consolidate all versions of the instrument and develop a prefinal version of the instrument for use in field testing; (5) pre-testing in a group of 98 nurses in operation rooms, hemodialysis rooms, emergency rooms, internist rooms, pulmonary rooms, surgery care rooms, medical checkups, and polyclinics. The final results of the HARS translation into Indonesian can be seen in table 2. pre-testing in a group of 98 nurses in operation room, hemodialysis room, emergency room, internist room, pulmonary room, surgery care room, medical checkup, and polyclinics. The final results of the translation into Indonesian can be seen in table 2. Statistical Analyses Data were analyzed by the Statistical Package for the Social Sciences (SPSS ver. 21, Chicago, IL, USA), in order to describe continuous and qualitative variables, mean, standard deviation (SD) and percentage frequency were used respectively. The minimum, maximum and variance were also reported for each item of the questionnaire. Validity and Reliability Pearson product moment correlation was used to evaluate the construct validity of each item to the total score. HAM-A test correlations were considered as ‘good to excellent’ when r ≥ 0.75, as ‘good’ when r ranged between 0.5 and 0.7, as ‘fair’ when r ranged between 0.25 and 0.50, and as ‘little or no relationship’ when r was less than 0.25 (Kline, 2000; Portney & Watkins, 2009; Terwee et al., 2007). Cronbach's alpha scores with split half method were used to assess the internal consistency reliability of the HAM-A questionnaire. A value below 0.70, the questionnaire is ‘unacceptable’ a value between 0.70 and 0.79 is considered ‘fair’, a value between 0.80 and 0.89 considered ‘good’, and a value 0.90 and above considered ‘excellent’ (Cicchetti, 1994; Nunnally & Bernstein, 1994; Michalopoulos et al. 2015; Taber, 2018). RESULTS Respondent characteristics The majority of respondents in this study were 20-30 years old (46.9%), the majority of respondents were women (77,6%), mostly married (82.7%), worked more than 5 to 10 years (45.9%) and mostly graduated from diploma III in nursing (80.6%) ( table 1). The result of validity and reliability test of HAM- A Indonesian version As is shown in table 3, the mean of the total HAM-A score was 10,58 (± 5,82). The 4th item on “Insomnia (difficulty in falling asleep, broken sleep, unsatisfying sleep and fatigue on waking, dreams, nightmares, night terrors)” showed the highest score (1,11, ± 0,73), whereas the 10th item on “Respiratory symptoms (pressure or constriction in chest, choking feelings, sighing, dyspnea)” had the lowest score (4,1 ± 0,64). The largest and smallest variance was also observed in item 2 and item 4 (0,53) and item 1 (0,37), respectively. The smallest Pearson correlation value is 0.529 (item number 11 on “Gastrointestinal symptoms (difficulty in swallowing, wind abdominal pain, burning sensations, abdominal fullness, nausea, vomiting, borborygmi, looseness of bowels, loss of weight, constipation)”, and the largest Pearson correlation is 0.727 (item number 6 on “Depressed mood: loss of interest, lack of pleasure in hobbies, Table 1. Characteristics of respondents (n=98) Characteristics n % Ages 20-30 46 46.9 >30-40 43 43.9 >40-50 9 9.2 Gender Male 22 22.4 Female 76 77.6 Marital Status Married 81 82.7 Not married 15 15.3 Ever married 2 2 Working Experience 0-5 35 35.7 >5-10 45 45.9 >10-15 13 13.3 >15-20 5 5.1 Level of Education Diploma III in nursing 79 80.6 Strata 1 in nursing 19 19.4 Total 98 100 I. M. RAMDAN 36 | pISSN: 1858-3598  eISSN: 2502-5791 depression, early waking, diurnal swing”). Based on the previous criteria, it can be concluded that all of the HAM-A items in Indonesian version are declared "fair" or in this study called as a valid because Pearson correlation (r) ranged between 0.5 and 0.7. If item question number 2 “Tension: Feelings of tension, fatigability, startle response, moved to tears easily, trembling, feelings of restlessness, inability to relax” is deleted this can increase Cronbach's alpha coefficient by 0.753, If item question number 12 “Genitourinary symptoms (frequency of micturition, urgency of micturition, amenorrhea, menorrhagia, development of frigidity, premature ejaculation, loss of libido, impotence)” is deleted this can increase Cronbach's alpha coefficient by 0.745 and if item question number 5 “Intellectual (difficulty in concentration, poor memory)” is deleted this can increase Cronbach's alpha coefficient by 0.744. To get the Indonesian version of HAM-A with the highest level of reliability, it is recommended that item number 2 “Tension: Feelings of tension, fatigability, startle response, moved to tears easily, trembling, feelings of restlessness, inability to relax” be omitted so that Cronbach's Alpha coefficient becomes 0.753. But in general, the combination of all 14 items of HAM-A has shown the fair reliability (or Table 2. The Hamilton Anxiety Rating Scale (HAM-A): Original (English)(Hamilton, 1959) and Translated (Indonesian) Version. Original version Indonesian version Item 1 Anxious mood (Worries, an anticipation of the worst, fearful anticipation, irritability) Perasaan cemas (merasa khawatir, firasat buruk, takut akan fikiran sendiri, lekas marah atau mudah tersinggung) Item 2 Tension (Feelings of tension, fatigability, startle response, moved to tears easily, trembling, feelings of restlessness, inability to relax) Ketegangan (merasa tegang, merasa lelah, respon yang mengejutkan, mudah meneteskan air mata, merasa gemetar, merasa gelisah, tidak mampu untuk bersantai) Item 3 Fears (Of dark, of strangers, of being left alone, of animals, of traffic, of crowds) Ketakutan (takut terhadap gelap, takut terhadap orang asing, takut ditinggalkan sendirian, takut pada hewan, takut pada keramaian lalu lintas, takut pada kerumunan orang banyak) Item 4 Insomnia (Difficulty in falling asleep, broken sleep, unsatisfying sleep and fatigue on waking, dreams, nightmares, night terrors) Insomnia (kesulitan tidur, tidur tidak memuaskan, merasa lelah saat bangun, mimpi buruk, terbangun tengah malam) Item 5 Intellectual (Difficulty in concentration, poor memory) Intelektual (sulit berkonsentrasi, sulit mengingat) Item 6 Depressed mood (Loss of interest, lack of pleasure in hobbies, depression, early waking, diurnal swing) Perasaan depresi (kehilangan minat, kurangnya kesenangan dalam hobi, perasaan bersedih, sering terbangun dini hari saat tidur malam) Item 7 Somatic (muscular) (Pains and aches, twitching, stiffness, myoclonic jerks, grinding of teeth, unsteady voice, increased muscular tone) Gejala somatik (otot) (nyeri atau sakit otot, kedutan, otot terasa kaku, gigi gemertak, suara tidak stabil, tonus otot meningkat) Item 8 Somatic (sensory) (Tinnitus, blurring of vision, hot and cold flushes, feelings of weakness, pricking sensation) Somatik (sensorik) (Telinga terasa berdenging, penglihatan kabur, muka memerah, perasaan lemah, sensasi ditusuk-tusuk) Item 9 Cardiovascular symptoms (Tachycardia, palpitations, pain in chest, throbbing of vessels, fainting feelings, missing beat) Gejala-gejala kardiovaskular (takikardi, palpitasi, nyeri dada, denyut nadi meningkat, perasaan lemas/lesu seperti mau pingsan, denyut jantung serasa berhenti sekejap) Item 10 Respiratory symptoms (Pressure or constriction in chest, choking feelings, sighing, dyspnea) Gejala pernapasan (nafas terasa sesak/dada terasa ditekan, perasaan tercekik, sering menarik nafas dalam, nafas pendek/tersengal-sengal) Item 11 Gastrointestinal symptoms (Difficulty in swallowing, wind abdominal pain, burning sensations, abdominal fullness, nausea, vomiting, borborygmi, looseness of bowels, loss of weight, constipation) Gejala gastrointestinal (kesulitan menelan, nyeri perut, perut terasa kembung, sensasi terbakar, perut terasa penuh, merasa mual, muntah, sukar buang air besar/BAB, kehilangan berat badan, konstipasi) Item 12 Genitourinary symptoms (Frequency of micturition, urgency of micturition, amenorrhea, menorrhagia, development of frigidity, premature ejaculation, loss of libido, impotence) Gejala genitourinari (frekuensi berkemih meningkat, tidak dapat menahan air seni, tidak datang bulan, darah haid lebih banyak dari biasanya, gairah sex menurun, ejakulasi dini, kehilangan libido, impotensi) Item 13 Autonomic symptoms (Dry mouth, flushing, pallor, tendency to sweat, giddiness, tension headache, raising of hair) Gejala otonom (mulut kering, muka kemerahan, muka pucat, sering berkeringat, merasa pusing, kepala terasa berat, merasa tegang, rambut terasa menegang) Item 14 Behavior (Fidgeting, restlessness or pacing, tremor of hands, furrowed brow, strained face, sighing or rapid respiration, facial pallor, swallowing, etc.) Tingkah laku (gelisah, tidak tenang/sering mondar-mandir, tangan gemetar, alis berkerut, wajah tegang, sering mendesah atau pernapasan cepat, wajah pucat, sering menelan ludah, dll.) JURNAL NERS http://e-journal.unair.ac.id/JNERS | 37 in this study called as a reliable) with Cronbach’s alpha coefficient of 0,756. This result generally shows the Indonesian version of HAM-A is reliable to measure work-related stress in nurses. DISCUSSION Validity expresses the degree to which a measurement measures what it purpose to measure. Validity tests are categorized into two broad components namely; internal and external validities. Internal validity refers to how accurately the measures obtained from the research was actually quantifying what it was designed to measure whereas external validity refers to how accurately the measures obtained from the study sample described the reference population from which the study sample was drawn (Bolarinwa, 2015). Reliability is the extent to which a measurement of a phenomenon provides stable and consist result (Taherdoost, 2016), and Cronbach's alpha is an accurate estimate of reliability and the Spearman-Brown formula is an accurate method to calculated reliability coefficient (Eisinga, Grotenhuis, & Pelzer, 2013). According to Ursachi, Horodnic, & Zait (2015), the Cronbach Alpha coefficient between 0.6-0.7 indicates an acceptable level of reliability, and 0.8 or greater a very good level. The finding of the present study indicates that the Indonesian version of the HAM-A has a high enough internal and external validity, which can reveal the causal relationship between independent and dependent variables related to work-related stress in nursing with generalized results. This can be seen from the correlation coefficient (r) of the Pearson Product Moment between the item score and the total score ranging from 0.529 (min) to 0.967 (max) with a significant positive correlation (p = 0.000). In accordance with Bryman's (2001 ) opinion that internal validity is common to refer to the factor that has a causal impact as the independent variable and the effect as the dependent variable, and Mc Dermot's (2011) opinion that external validity refers to the generalizability of findings from a study, or the extent to which conclusions can be applied across different populations, settings, treatments, and outcomes In this study, the English version of HAM-A was translated into Indonesian language and the reliability and validity of the Indonesian version of the HAM-A were investigated using a representative sample of nurses from various aspect i.e department/care rooms, age, marital status, educational level, and work experience, the finding indicates that the HAM-A in Indonesian version has satisfactory psychometric properties with adequate validity and reliability, so that it can be used to measure work-related stress on nurses. Similar to this study, translation of HAM-A into another language version has been done several times and getting valid and reliable results. In clinical research settings, HAM-A is a reliable and valid measure for the assessment of global anxiety in the adolescent population (Clark & Donovan, 1994); HAM-A are reliable and valid instruments that can be used among end-stage renal disease (ESRD) patients undergoing hemodialysis (HD)(Gencoz, Gencoz, & Soykan, 2007); HAM-A is a valid and reliable instrument for the assessment of depression in the Urdu language (Hashmi, Naz, Asif, & Khawaja, 2016); In Indian language with video recorded interview, HAM-A inter-rater reliability has found excellent to asses patients with major depressive disorder (MDD) (Prasad et al., 2009). In general/workers setting, this study supports the results of previous studies, among others: HARS in the Arabic version is valid and reliable to measure work-related stress among working women in Gaza Strip (Aqel & Thabet, 2017), the HAM-A showed good internal consistency to assess the Romanian employees working in non-governmental organizations (Craiovan, 2015), HAM-A can be used globally and is valid and reliable to measure work- related stress on students (Gupta et al., 2014), HAM- A has a high-reliability index to measure anxiety oncology nursing in Athens, Greece (Karanikola et al., 2016) and the HARS in Greek language was reliable to assess work-related stress in emergency nursing Table 3. Descriptive Characteristics and the Pearson Correlation of Each Data for Internal Consistency of Indonesian Version of the HAM-A Questionnaire (n=98) Item Mean SD Pearson correlation (r) p Corrected item/total correlation Cronbach’s alpha if item deleted Item 1 0.64 0.613 0.599 0.000 0.586 0.743 Item 2 0.96 0.731 0.697 0.000 0.702 0.753 Item 3 0.66 0.657 0.537 0.000 0.574 0.742 Item 4 1.11 0.731 0.600 0.000 0.587 0.740 Item 5 0.87 0.715 0.550 0.000 0.565 0.744 Item 6 0.86 0.603 0.727 0.000 0.750 0.743 Item 7 0.88 0.703 0.672 0.000 0.673 0.738 Item 8 0.72 0.662 0.629 0.000 0.640 0.738 Item 9 0.46 0.715 0.629 0.000 0.702 0.738 Item 10 0.41 0.645 0.562 0.000 0.651 0.741 Item 11 0.83 0.589 0.529 0.000 0.618 0.741 Item 12 0.82 0.626 0.693 0.000 0.526 0.745 Item 13 0.82 0.648 0.569 0.000 0.714 0.737 Item 14 0.55 0.628 0.569 0.000 0.562 0.743 Total score 10.58 5.82 1 - - I. M. RAMDAN 38 | pISSN: 1858-3598  eISSN: 2502-5791 personnel in Greece (Stathopoulou, Karanikola, Panagiotopoulou, & Papathanassoglou, 2011). CONCLUSION The result of the study demonstrates that the Indonesian version of the HAM-A fulfils the criteria of a reliable (fair acceptable criteria) and valid (good criteria) assessment tool to assess the work-related stress in the nursing profession. This scale showed good psychometric properties in the nursing profession with different education, gender, work experience, and different department. The high internal consistency and construct validity support the application of the HAM-A as an easy-administered tool to asses work-related stress in the nursing profession. ACKNOWLEDGMENTS The authors are very grateful to the Hospital management for giving research permission, all nurses participated in this study, and expert committee (translators, epidemiologist and health professionals expert). REFERENCES Aqel, N. A., & Thabet, A. (2017). Work and Family Stressors , Depression , and Anxiety among Working Women in Gaza Strip. Global Journal of Intellectual & Develompental Disabilities (GJIDD), 555615(November), 001–006. Beaton, D. E., Bombardier, C., Guillemin, F., Ferraz, M. B., Borsa, J. C., Wamser, G. H., … Schoemaker, M. M. (2000). Guidelines for the Process of Cross- Cultural Adaptation of Self-Report Measures. Spine, 25(24), 3186–3191. https://doi.org/10.1111/cch.12124 Bolarinwa, O. A. (2015). Principles and methods of validity and reliability testing of questionnaires used in social and health science researches. Nigerian Postgraduate Medical Journal, 22(4), 195–201. doi: 10.4103/1117-1936.173959 Bryman, A. (2001). Social Research Methods. New York: Oxford University Press Inc. Cicchetti, D. V. (1994). Guidelines, Criteria, and Rules of Thumb for Evaluating Normed and Standardized Assessment Instruments in Psychology. Psychological Assessment, 6(4), 284– 290. https://doi.org/10.1037/1040-3590.6.4.284 Clark, D. B., & Donovan, J. E. (1994). Reliability and Validity of the Hamilton Anxiety Rating Scale in an Adolescent Sample. Journal of the American Academy of Child & Adolescent Psychiatry, 33(3), 354–360. doi: 10.1097/00004583-199403000- 00009 Craiovan, P. M. (2015). Burnout, Depression and Quality of Life among the Romanian Employees Working in Non-governmental Organizations. Procedia - Social and Behavioral Sciences, 187, 234–238. https://doi.org/10.1016/j.sbspro.2015.03.044 Deguchi, Y., Iwasaki, S., Ishimoto, H., Ogawa, K., Fukuda, Y., Nitta, T., … Inoue, K. (2017). Relationships between temperaments, occupational stress, and insomnia among Japanese workers. PLoS ONE, 12(4), 1–13. https://doi.org/10.1371/journal.pone.0175346 Eisinga, R., Grotenhuis, M. Te, & Pelzer, B. (2013). The reliability of a two-item scale: Pearson, Cronbach, or Spearman-Brown? International Journal of Public Health, 58(4), 637–642. https://doi.org/10.1007/s00038-012-0416-3 Fan, L. B., Blumenthal, J. A., Watkins, L. L., & Sherwood, A. (2015). Work and home stress: Associations with anxiety and depression symptoms. Occupational Medicine, 65(2), 110–116. https://doi.org/10.1093/occmed/kqu181 Gencoz, F., Gencoz, T., & Soykan, A. (2007). Psychometric properties of the Hamilton Depression Rating Scale and other physician-rated psychiatric scales for the assessment of depression in ESRD patients undergoing hemodialysis in Turkey. Journal Psychology, Health & Medicine, 12(4), 450–459. doi: 10.1080/13548500600892054 Gupta, R., Punia, A., Mathur, S., Gupta, R., Kawatra, A., & Verma, K. (2014). Assessment of psychological stress in first year students of a private Indian Journal of Basic and Applied Medical Research. Indian Journal of BAsic and Applied MEdical Research, 1956(March), 437–443. Hamilton, M. (1959). The assessment of anxiety states by rating. The British Journal of Medical Psychology, 32(1), 50–55. DOI: 10.1111/j.2044- 8341.1959.tb00467.x Hashmi, A. M., Naz, S., Asif, A., & Khawaja, I. S. (2016). Urdu translation of the hamilton rating scale for depression: Results of a validation study. Pakistan Journal of Medical Sciences, 32(6), 1479–1483. https://doi.org/10.12669/pjms.326.11399 Huerta-Franco, M.-R. (2013). Effects of occupational stress on the gastrointestinal tract. World Journal of Gastrointestinal Pathophysiology, 4(4), 108. https://doi.org/10.4291/wjgp.v4.i4.108 International Labour Organization. (2016). Workplace Stress. World Day for Safety and Health at Work. Turin - Italy: International Labour Organization. https://doi.org/10.1017/CBO9781107415324.0 04 Karanikola, M. N. K., Giannakopoulou, M., Kalafati, M., Kaite, C. P., Patiraki, E., Mpouzika, M., … Middleton, N. (2016). Anxiety symptoms and quality of interaction among oncology nurses: A correlational, cross-sectional study. Revista Da Escola de Enfermagem, 50(5), 800–807. https://doi.org/10.1590/S0080- 623420160000600013 Katherine Shear, M., Bilt, J. Vander, Rucci, P., Endicott, J., Lydiard, B., Otto, M. W., … Frank, D. M. (2001). Reliability and validity of a Structured Interview Guide for the Hamilton Anxiety Rating Scale (SIGH-A). Depression and Anxiety, 13(4), 166–178. https://doi.org/10.4103/1117-1936.173959 https://doi.org/10.1097/00004583-199403000-00009 https://doi.org/10.1097/00004583-199403000-00009 https://doi.org/10.1080/13548500600892054 https://doi.org/10.1111/j.2044-8341.1959.tb00467.x https://doi.org/10.1111/j.2044-8341.1959.tb00467.x JURNAL NERS http://e-journal.unair.ac.id/JNERS | 39 https://doi.org/10.1002/da.1033 Khamisa, N., Peltzer, K., Ilic, D., & Oldenburg, B. (2017). Effect of personal and work stress on burnout, job satisfaction and general health of hospital nurses in South Africa. Health SA Gesondheid, 22, 252–258. https://doi.org/10.1016/j.hsag.2016.10.001 Kline, P. (2000). The Handbook of Psychological Testing (Second). London and New York: Routledge. La Torre, G., Sestili, C., Mannocci, A., Sinopoli, A., De Paolis, M., De Francesco, S., … De Giusti, M. (2018). Association between Work Related Stress and Health Related Quality of Life: The Impact of Socio- Demographic Variables. A Cross Sectional Study in a Region of Central Italy. International Journal of Environmental Research and Public Health, 15(1), 159. https://doi.org/10.3390/ijerph15010159 Li, J., Loerbroks, A., Bosma, H., & Angerer, P. (2016). Work stress and cardiovascular disease: A life course perspective. Journal of Occupational Health, 58(2), 216–219. https://doi.org/10.1539/joh.15-0326-OP López-Pina, J. A., Sánhez-Meca, J., & Rosa-Alcázar, A. I. (2009). The Hamilton Rating Scale for Depression: a meta analytic reliability generalization study. International Journal of Clinical and Health Psychology, 9(9), 143–159. Maier, W., Buller, R., Philip, M., & Heuser, I. (1988). The Hamilton Anxiety Rating Scale: Reliability, validity and sensitivity to change in anxiety and depresive disorders. J Affect Disord, 14(1), 61–68. DOI:10.1016/0165-0327(88)90072-9 McDermott, R. (2011). Internal and external validity. In J. N. Druckman, D. P. Green, J. H. Kuklinski, & A. Lupia (Eds.), Cambridge handbook of experimental political science. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511921452.0 03 Michalopoulos, L. T. M., Murray, L. K., Kane, J. C., Van Wyk, S. S., Chomba, E., Cohen, J., … Bolton, P. A. (2015). Testing the validity and reliability of the Shame Questionnaire among sexually abused girls in Zambia. PLoS ONE, 10(4), 1–17. https://doi.org/10.1371/journal.pone.0123820 Michie, S. (2002). Causes and management of stress at work. Occupational Environ Med, 59, 67–72. https://doi.org/10.1136/oem.59.1.67 Moore, S., Sikora, P., Grunberg, L., & Greenberg, E. (2007). Work Stress and Alcohol Use: Examining the Tension-Reduction Model as a Function of Worker’s Parent’s Alcohol Use. Addict Behav, 32(12), 3114–3121. https://doi.org/10.1016/j.immuni.2010.12.017.T wo-stage Moustaka, E., & Constantinidis, T. C. (2010). Sources and effects of work-related stress in nursing. Health Science Journal, 4(4), 210–216. Nafeesa, M., Vidhya, V., Vijayalakshmi, P., & Rajkumar, P. (2017). Annals of Musculoskeletal Disorders Central Perceived Work-Related Psychosocial Stress and Musculoskeletal Disorder Complaints among Call Centre Workers in India – A Cross Sectional Study. Ann Musc Disord, 1(1), 1–6. Retrieved from https://www.jscimedcentral.com/Musculoskelet alDisorders/musculoskeletaldisorders-1- 1005.pdf Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric Theory (3rd ed.). USA: McGraw-Hill, Inc. https://doi.org/10.1177/014662169501900308 Obradović, J., & Čudina-Obradović, M. (2013). Work Stress and Marital Quality in Dual Earner Couples: A Test of Three Mediation Models. Drustvena Istrazivanja, 22(4), 673–691. https://doi.org/10.5559/di.22.4.06 Piko, B. F. (2006). Burnout, role conflict, job satisfaction and psychosocial health among Hungarian health care staff: A questionnaire survey. International Journal of Nursing Studies, 43(3), 311–318. https://doi.org/10.1016/j.ijnurstu.2005.05.003 Pillay, R. (2009). Work satisfaction of professional nurses in South Africa: A comparative analysis of the public and private sectors. Human Resources for Health, 7(15), 1–10. https://doi.org/10.1186/1478-4491-7-15 Portney, L., & Watkins, M. (2009). Foundations of clinical research: applications to practice Pearson education. New Jersey: Upper Saddle River. Prasad, K. M., Udupa, K., Kishore, K., Thirthalli, J., Sathyaprabha, T., & Gangadhar, N. (2009). Inter- rater reliability of Hamilton depression rating scale using video-recorded interviews — Focus on rater-blinding. Indian Journal of Psychiatry, 51(3), 191–194. doi: 10.4103/0019-5545.55085 Rose, D. M., Seidler, A., Nübling, M., Latza, U., Brähler, E., Klein, E. M., … Beutel, M. E. (2017). Associations of fatigue to work-related stress, mental and physical health in an employed community sample. BMC Psychiatry, 17(1), 1–8. https://doi.org/10.1186/s12888-017-1237-y Singh, A. S., & Masuku, M. B. (2014). Sampling Techniques & Determination of Sample Size in Applied Statistics Research: An Overview. International Journal of Economics, Commerce and Management, 2(11), 1–22. doi: 10.12691/jpar-3- 1-8. Stathopoulou, H., Karanikola, M., Panagiotopoulou, F., & Papathanassoglou, E. (2011). Anxiety levels and related symptoms in emergency nursing personnel in Greece. J Emerg Nurs, 37(4), 314– 320. doi: 10.1016/j.jen.2010.03.006 Taber, K. S. (2018). The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education. Research in Science Education, 48(6), 1273–1296. https://doi.org/10.1007/s11165-016-9602-2 Taherdoost, H. (2016). Validity and Reliability of the Research Instrument ; How to Test the Validation of a Questionnaire / Survey in a Research. International Journal of Academic Research InManagement (IJARM), 5(3), 28– https://doi.org/10.1016/0165-0327(88)90072-9 https://doi.org/10.1017/CBO9780511921452.003 https://doi.org/10.1017/CBO9780511921452.003 https://doi.org/10.1177%2F014662169501900308 https://doi.org/10.4103/0019-5545.55085 https://doi.org/10.1016/j.jen.2010.03.006 I. M. RAMDAN 40 | pISSN: 1858-3598  eISSN: 2502-5791 36. http://dx.doi.org/10.2139/ssrn.3205040 Terwee, C. B., Bot, S. D. M., de Boer, M. R., van der Windt, D. A. W. M., Knol, D. L., Dekker, J., … de Vet, H. C. W. (2007). Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology, 60(1), 34–42. https://doi.org/10.1016/j.jclinepi.2006.03.012 Thompson, E. (2015). Hamilton rating scale for anxiety (HAM-A). Occupational Medicine, 65(7), 601. https://doi.org/10.1093/occmed/kqv054 Ursachi, G., Horodnic, I. A., & Zait, A. (2015). How Reliable are Measurement Scales? External Factors with Indirect Influence on Reliability Estimators. Procedia Economics and Finance, 20(15), 679–686. https://doi.org/10.1016/s2212-5671(15)00123- 9 Vernekar, S. P., & Shah, H. (2018). A study of work- related stress among nurses in a tertiary care hospital in Goa. International Journal of Cummunity Medicine and Public Health, 5(2), 657– 661. doi: http://dx.doi.org/10.18203/2394- 6040.ijcmph20180246 Vignoli, M., Muschalla, B., & Mariani, M. G. (2017). Workplace Phobic Anxiety as a Mental Health Phenomenon in the Job Demands-Resources Model. BioMed Research International, 1–10. https://doi.org/10.1155/2017/3285092 World Health Organization (WHO). (2007). Stress at the workplace. Geneva. Retrieved from http://www.who.int/occupational_health/topics /stressatwp/en https://dx.doi.org/10.2139/ssrn.3205040 http://dx.doi.org/10.18203/2394-6040.ijcmph20180246 http://dx.doi.org/10.18203/2394-6040.ijcmph20180246