













	
                      
	
                                           

	Abstract
	Introduction
	Methods
	Results
	Discussion
	Conclusion
	Acknowledgements
	References



About the Author(s)



Clement N. Kufe   
Department of Epidemiology and Surveillance, National Institute for Occupational Health, National Health Laboratory Service, Johannesburg, South Africa
School of Health Systems and Public Health, Faculty of Health Sciences, University of Pretoria, Pretoria, South Africa


Colleen Bernstein  
Department of Psychology, School of Human and Community Development, University of the Witwatersrand, Johannesburg, South Africa


Kerry Wilson  
Department of Epidemiology and Surveillance, National Institute for Occupational Health, National Health Laboratory Service, Johannesburg, South Africa
School of Public Health, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa


Citation


 Kufe, C.N., Bernstein, C., & Wilson, K. (2024). Reliability, validity and dimensionality of the 12-Item General Health Questionnaire among South African healthcare workers. African Journal of Psychological Assessment, 6(0), a144. https://doi.org/10.4102/ajopa.v6i0.144

Note: Additional supporting information may be found in the online version of this article as Online Appendix 1.






                          
                               
                                    
                                            
                                                    

	
	

Original Research



	

Reliability, validity and dimensionality of the 12-Item General Health Questionnaire among South African healthcare workers


	 Clement N. Kufe, Colleen Bernstein, Kerry Wilson
Received: 14 June 2023; Accepted: 03 Mar. 2024; Published: 17 Oct. 2024


Copyright: © 2024. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution,
and reproduction in any medium, provided the original work is properly cited.



	
Abstract


Healthcare workers (HCWs) were among the high-risk groups for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and suffer a high burden of challenges with mental health including depression, anxiety, traumatic stress, avoidance and burnout. The 12-Item General Health Questionnaire (GHQ-12) has shown the best fit in both a one-factor structure and a multidimensional structure for the screening of common mental disorders and psychiatric well-being. The aim was to test for reliability and validity and ascertain the factor structure of the GHQ-12 in a South African HCW population. Data were collected from 832 public hospital and clinic staff during the coronavirus disease 2019 (COVID-19) pandemic in Gauteng, South Africa. The factor structure of the GHQ-12 in this professional population was examined by exploratory factor analysis (EFA) to identify factors, confirmatory factor analysis (CFA) for construct validity and structural equation modelling (SEM) to establish model fit. The GHQ-12 median score was higher (n = 25) in women than in men (n = 24), p = 0.044. The assumptions for inferential statistics were tested: the determinant for the correlation matrix was 0.034, Bartlett’s test of sphericity was p < 0.001, Chi-square 2262.171 and the Kaiser-Meyer-Olkin (KMO) of sampling adequacy was 0.877. The four factors identified were labelled as social dysfunction (37.8%), anxiety depression (35.4%), capable (24.9%) and self-efficacy (22.7%). The sample had a Cronbach’s alpha and McDonald’s Omega coefficient of 0.85.

Contribution: The study highlighted the gaps in the use of GHQ-12. The findings affirm the validity and reliability of the GHQ-12 in this group of professionals and the multidimensional structure for screening for psychological distress.

Keywords: healthcare workers; reliability; validity; 12-Item General Health Questionnaire; dimensionality.


Introduction


The coronavirus disease 2019 (COVID-19) arose in China in December 2019 and by early February 2020 reports of psychological pressures on Chinese healthcare workers (HCWs) were published (Kang et al., 2020). The coronavirus disease 2019 did not only affect the mental health of Chinese HCWs but also their physical health as it rapidly spread throughout the world. By the 11th of March, the World Health Organization (WHO) had declared COVID-19 a global pandemic (Shaukat et al., 2020). The first recognised COVID-19 case was diagnosed in South Africa on 05 March 2020 and 116 cases were registered by 18 March 2020. Following the National State of Disaster declared on the 15th of March, with schools closing on the 18 March 2020, a countrywide lockdown began on the 26 March 2020 in South Africa. South African HCWs were designated essential services and continued working at the forefront in the fight against the COVID-19 pandemic. Healthcare workers were responsible for screening, testing and managing COVID-19-positive patients as well as unknown COVID-19 patients, along with those responsible for cleaning healthcare facilities and, dealing with the administration and management of facilities (Lee et al., 2022). A high degree (57.4%) of psychological distress and a strong association between perceived risks associated with the presence of COVID-19 in the healthcare workplace and associated psychological distress was reported among HCWs in South Africa throughout the pandemic (Lee et al., 2022).

Epidemic outbreaks have been known to place an unprecedented demand on HCWs resulting in increased deaths, infection, shortage of medication and vaccines, increased workload and a lack of personal protective equipment, and feelings of inadequate support all exacerbating the mental distress and burden of HCWs (Chen et al., 2020; Lai et al., 2020; Lee et al., 2022). A systematic review of the mental health of HCWs during the COVID-19 pandemic reported the lowest prevalence of anxiety, depression and stress as 24.1%, 12.1% and 29.8% in HCWs from Fujian province in China and the highest prevalence of 67.6%, 55.9% and 62.9% in Spain for cross-sectional studies published in English (Vizheh et al., 2020). Another review of studies at the time identified risk factors for COVID-19-related health impact in HCWs being: inadequate hand hygiene, high-risk department, diagnosed family member, suboptimal hand hygiene before and after contact with patients, improper personal protective equipment use, close contact with patients (≥ 12 times/day), long daily contact hours (≥ 15 h) and female HCWs disproportionated affected (Shaukat et al., 2020).

Because of stigma, HCWs were likely to experience psychological distress in silence, which may have led to an increased risk of suicidal ideation (Dutheil et al., 2019; Knaak et al., 2017; Mehta et al., 2018). The uneasy climate during COVID-19 characterised by panic and fear in the general population was likely increased in HCWs as they needed to manage patient care at a time when their own physical and mental well-being was at also stake (WHO, 2020). The unremitting stress of the pandemic or epidemic could have led HCWs to experience elevated levels of anxiety, fear, panic attacks, posttraumatic stress symptoms, psychological distress, stigma, avoidance of contact, depressive symptoms, sleep disturbances, feelings of helplessness, interpersonal isolation from family social support, and concern regarding contagion exposure to their friends and family (Jawad et al., 2021; Rana et al., 2020; Søvold et al., 2021). At the same time, some programmes put in place faced resistance from HCWs in acknowledging the experience of psychological difficulties (Buselli et al., 2021). Therefore, a need to provide evidence-informed data on the mental health and well-being effects of HCWs to mitigate these challenges and provide evidence-informed recommendations for the future as the world transitions out of the pandemic (Søvold et al., 2021).

The General Health Questionnaire (GHQ) developed in the 1970s was initially intended for use as a unidimensional tool of 60-items to describe the risk of mental disorders. Many shortened versions of the GHQ were developed such as the GHQ-30, the GHQ-28, the GHQ-20 and the GHQ-12 (Burvill & Knuiman, 1983; Goldberg, 1972, 1989; Illness et al., 1972), all of which have been subject to factor analytic procedures to identify whether each of the scaled versions provide additional utility (Kline, 2014). The shorter GHQ-12 was used in epidemiologic studies, the majority of which and subsequent factor analytic studies failed to accept the unitary construct (Corti, 1994) and proposed a multidimensional structure (Werneke et al., 2000).

Because of its brevity, the 12-item GHQ has been widely used as a screening instrument for the presence of mental disorders in many countries (Donath, 2001; Doyle et al., 2012; Hu et al., 2007; Illness et al., 1972; Lundin et al., 2016; Özdemir & Rezaki, 2007; Uras et al., 2012). While the GHQ has been widely used to date, no study could be identified that had examined the factor structure on a South African sample.

The factor analytic approach assesses interrelationships within a set of variables to construct a smaller number of hypothetical variables or factors that contain the essential information of the larger set of observed variables, thus reducing the overall complexity of the data by taking advantage of the inherent interdependencies (Campbell et al., 2003). Factor analysis is useful for measuring constructs that are not readily observable, summarising large observations into smaller numbers and providing evidence of construct validity by hypothesis testing (Kline, 2014). Studies that have been carried out to assess the mental health of HCWs including the use of the 12-Item General Health Questionnaire (GHQ-12) have not been validated in all professional populations. This study subsequently aimed to assess the validity and reliability and determine the factor structure of the GHQ-12 in the South African, Gauteng, healthcare worker population.


Methods


Study design

A cross-sectional study was conducted in three public tertiary hospitals, two regional hospitals, two community health centres and nine clinics in Gauteng, South Africa between August and October 2020.

Participants

Data were collected from a convenience sample of 832 male and female hospital staff during the COVID-19 pandemic. The median (25th–75th percentile) age of the respondents was 44 (34–54) years, 81.7% identified as female, 17.9% as male and 0.4% as another. Furthermore, 99% were South African citizens and the remainder were from Southern Africa. The most common home language was Venda, followed by Setswana and IsiZulu. Majority (33.6%) of the participants were nurses, 11% were medical doctors, 8.8% were radiologist, 2.4% were either physiotherapists or pharmacists, 2.6% were clerks, 10.4% were working in the delivery department and the remaining 30.2% were involved in a variety of functions.

Instruments

A self-administered GHQ-12 hard copy and electronic questionnaire exploring socio-demographic information, assessing stress and psychological effects, perceptions, attitudes and behaviour around COVID-19 was used. The GHQ-12 consisted of 12 statements to which respondents indicated agreement on a four-point scale (1 = Better than usual, 2 = Same as usual, 3 = Worse than usual and 4 = Much worse than usual) on mental stress (Goldberg, 1972).

Procedure

The GHQ-12 and others were provided to the Gauteng Department of Health, Central Office, and Wellness Unit from where it was distributed to the Wellness Coordinators of the Johannesburg facilities. The questionnaire was printed and the Wellness coordinators distributed it to HCW from all the job categories and collected the completed questionnaires. The returned questionnaires were sent to the National Institute for Occupational Health (NIOH) for capturing and analysis. The online version of the questionnaire was available on Research Electronic Data Capture (REDCap), and the link was provided to the coordinators. Data were captured on REDCap (Harris et al., 2009).

Data analysis

Data were analysed using Stata version 16.1/MP (StataCorp, College Station, TX, United States [US]) after exploration to establish missing data points. The univariate normality of the items was assessed using the skewness and kurtosis of the responses. To establish the interrelatedness of the items for internal consistency and adequacy, an acceptable Cronbach’s alpha and a McDonald Omega of at least 0.70 was adopted. McDonald’s omega is unbiased for congeneric items with uncorrelated errors unlike Cronbach’s alpha (Malkewitz et al., 2023; Padilla & Divers, 2016). To ascertain whether the data satisfied the assumptions for inferential analysis, the Bartlett’s test supplemented by the Kaiser-Meyer-Olkin (KMO) measure of sample adequacy was used for the factor analysis with significant results and a KMO > 0.50, respectively.

Both exploratory and confirmatory factor analyses were implemented. Factor analysis was used to examine the theory-driven factor structure as evidential criteria using exploratory factor analysis (EFA) (Gorsuch, 1983; Lloret et al., 2017; Watkins, 2018) and by testing the assumptions using confirmatory factor analysis (CFA) to establish model fit (Loehlin, 2017; Russell, 2002; Young & Pearce, 2013). The use of EFA permitted a compromise by balancing the parsimony and comprehensiveness in the model that contains just enough factors to explain the important variations in the measured variables and later using multiple methods to advise on the plausible and appropriate factor solutions (Fabrigar et al., 1999; Gorsuch, 1983; Hair et al., 2010; Henson & Roberts, 2006; Lloret et al., 2017; Loehlin, 2017; Norris & Lecavalier, 2010). Using EFA, the appropriate factors to explain the relationship between observed variables were established. Orthogonal (varimax) was employed to determine the number of factors to retain by using the following criteria: (1) eigenvalues of the factors ≥ 1, (2) Cattell’s scree test, (3) internal consistency and (4) factors that yielded meaningful psychological constructs. Structural equation modelling (SEM) assessed the predictive utility of the factors identified with CFA by using maximum likelihood estimation for the following goodness-of-fit criteria: Likelihood ratio, population error, information criteria, baseline comparison and size of residuals (Hooper et al., 2008).

Assessment of model fit, comparison and predictive utility

Confirmatory factor analysis was used to ascertain whether the factor structure that was developed from the data matched the chosen conceptual models and an analysis goodness-of-fit test between the data and the model (Reymont & Jvreskog, 1993). Model fit was evaluated by examining the size and statistical significance of the factor loadings as well as the comparative fit index (CFI), Tucker-Lewis index (TLI), the standardised root mean square residual (SRMR) and the root mean square error of approximation (RMSEA) at 90% confidence interval (CI).

An RMSEA of 0.01, 0.05 and 0.08 is generally considered to represent excellent, good and mediocre fit, respectively, at 90% CI (MacCallum et al., 1996) while an SRMR of less than 0.08 is typically considered to be a very good fit while 0.05–1.0 was considered an acceptable fit (Hu & Bentler, 1999). A CFI and TLI close to 0.95 represents a good fit between the hypothesised model and the data while values in the range of 0.90–0.95 are acceptable (Hu & Bentler, 1999). Kline, 2015; McDonald & Ho, 2002). Competing models were compared by use of the Akaike’s information criterion (AIC), the Bayesian information criterion (BIC) and the likelihood ratio test where the lower values for AIC and BIC indicate a better fit in tandem with the parsimony goodness-of-fit index having the highest overall coefficient of determination (CD) or R-squared (Hooper et al., 2008).

Ethical considerations

Ethical approval to conduct the study was obtained from the University of the Witwatersrand, Human Research Ethics Committee (Medical) Clearance (certificate no.: M2006103). A wellness coordinator in each health facility invited the HCWs who provided consent and were enrolled in the study. Written consent was obtained from all participants before completing the anonymous online questionnaire.


Results


The GHQ-12 median score was 25 for females as compared to 24 for males and the difference was statistically significant, p = 0.044. The determinant for the correlation matrix was 0.034, the Bartlett’s test of sphericity was significant (p < 0.001) with a Chi-square of 2262.171 and the KMO of sampling adequacy of 0.877 confirming the suitability of data for the factor analytic technique. The entire sample had a general average inter-item covariance of 0.33 and a Cronbach’s alpha of 0.85, 0.85 for males and 0.84 for females illustrating satisfactory internal consistency in both groups and in males and females, respectively. For the McDonald’s omega coefficient, it was 0.85 for both males and females, 0.86 for males and 0.84 for females. The range of the item-scale correlation was 0.50 for ‘playing a useful part in things’ to 0.69 for the ‘unhappy’ and ‘depressed’ in the entire sample (Table 1). The values of Cronbach’s alpha of 0.85 and the McDonald’s omega of 0.85 were comparative or comparable. The majority of respondents completed all 12 questions, although ‘Been able to enjoy normal day-to-day activities’ was the most skipped question.



	TABLE 1: Item-scale analysis of the 12-Item Global Health Questionnaire.






Reliability, validity and the factor structure

For the factor analysis, 672 participants were included, 4 factors were retained, and all had a coefficient greater than 0.30. The test of internal consistency represented by a scale of reliability coefficient was 0.85 for all the 12 items, 0.78 for Factor I, 0.76 for Factor II, 0.64 for Factor III and 0.64 for Factor IV in orthogonal (varimax) rotation. The data satisfied the assumptions for inferential analysis in this population. The factor loadings were further labelled as follows: Factor I labelled ‘Social Dysfunction’ (37.8%); Factor II called ‘Anxiety and Depression’ (35.4%); Factor III called ‘Self-Efficacy’ (22.7%) and Factor IV labelled ‘Capable’ (24.9%) with the first three factors accumulatively representing 98.1% (Table 2).



	TABLE 2: Maximum likelihood estimates for orthogonal (varimax) factor loadings of the 12-Item General Health Questionnaire.






Exploratory factor analysis using orthogonal (varimax) to determine the number of factors. The four-factor structure labelled as Factor I (Social Dysfunction), Factor II (Anxiety and Depression), Factor III (Self-Efficacy) and Factor IV (Capable).

We examined the size and statistical significance of the factor loadings and common goodness-of-fit statistics. A unidimensional model was tested for all the 12 items in Model 1 named ‘Mental distress’. In Model 2, a four-factor structure was tested and labelled as ‘Social dysfunction’ (4 items), ‘Anxiety and Depression’ (4 items), ‘Self-Efficacy’ (2 items) and ‘Capable’ (2 items). While in Model 3, the four-factor structure was maintained but item distribution was ‘Social dysfunction’ (3 items), ‘Anxiety and Depression’ (3 items), ‘Self-Efficacy’ (3 items) and ‘Capable’ (3 items). Structural equation modelling assessed the predictive utility of the factors identified with CFA by using goodness-of-fit criteria (Table 3). The best fitting model was Model 3 with the most acceptable values (Figure 1) and the highest overall CD or R-squared as 0.972 (Online Appendix 1, Supplementary files 1).



	
	
	
FIGURE 1: The different factor models for the 12-Item General Health Questionnaire compared in South African healthcare workers.











	TABLE 3: Goodness-of-fit statistics for tested models of the 12-Item General Health Questionnaire in healthcare workers from South Africa.






The retained results from the SEM are presented in Model 3 of Figure 1. ‘Social dysfunction’ was positively and statistically significantly associated with ‘lost much sleep over worry’, ‘felt constantly under strain’, ‘felt that you could not overcome your difficulties’ and ‘been able to concentrate on what you are doing’. ‘Anxiety and Depression’ was positively and strongly associated with ‘been feeling unhappy and depressed’, ‘been losing confidence in yourself’ and ‘been thinking of yourself as a worthless person’. ‘Self-Efficacy’ was positively and statistically significantly associated with ‘felt that you are playing a useful part in things’ and ‘felt capable of making decisions about things’. ‘Capable’ was associated and statistically significant with ‘been able to enjoy your normal day-to-day activities’ and ‘been able to face up to your problems’.


Discussion


This was the first evaluation of the factor structure of the GHQ-12 in a South African population that found good internal consistency for the GHQ-12, reliability and validity of the GHQ-12 among HCWs. The EFA identified the factors and the CFA established the factor structure. The factor structure was not unitary (one factor distress) but a multidimensional scale made up of four factors: Social Dysfunction (Factor I), Anxiety and Depression (Factor II), Self-Efficacy (Factor III) and Capable (Factor IV) with a robust external validity for all four factors identified.

This study demonstrated adequate reliability and validity for a professional South African population and demonstrated the best-fit data with multidimensional structure of four factors similar to a multicentre study involving Ethiopian populations made up of Social Dysfunction, Anxiety and Depression, Self-Efficacy and Capable (Gelaye et al., 2015) and to the literate Kenyan study (Abubakar & Fischer, 2012).

The Cronbach’s alpha and the McDonald Omega coefficient are measures of the composite reliability for internal consistency of the GHQ-12 demonstrated a reliability higher than 0.70 and the two estimators were compared in this study. Similar results were found in another study consisting of a random sample of Brazilian physicians registered in the medical Council system (Oliveira et al., 2023) and a similar Cronbach’s alpha was found in primary care patients from Indonesians (Anjara et al., 2020).

Unlike the unidimensional factor structure reported in an Indonesian population (Anjara et al., 2020), the bi-factor structure in the Brazilian study (Oliveira et al., 2023) and the Norwegian Navy (Hystad & Johnsen, 2020), this study suggested a four-factor structure similar to the multidimensionality reported in the Spanish population (Sánchez-López & Dresch, 2008). The SEM conducted in this study did not include insomnia and mental health problems (HSCL-25) as in unidimensional domains (Hystad & Johnsen, 2020; Skogen et al., 2017) but rather opted for the multidimensionality as obtained from the initial results of the EFA and CFA.

This study showed a four-factor model made up of Social Dysfunction (Factor I), Anxiety and Depression (Factor II), Self-Efficacy (Factor III) and Capable (Factor IV) as the best explanation of a sample of South African HCWs compared to the three-factor model labelled as Anxiety-Depression, Social Dysfunction and Loss of Confidence when testing for the six-factor analytic models (Shevlin & Adamson, 2005) and reported in longitudinal and cross-sectional studies (El-Metwally et al., 2018; Liang et al., 2016; Mäkikangas et al., 2006). This study confirmed the valid use of GHQ-12 in a professional occupational group as previously shown in a study of young civil servants in China, but differed in the suggested factor structure, which was three-factor compared to the four-factor structure in this study (Liang et al., 2016).

A Finnish study using GHQ-12 and GHQ-20 concluded that the GHQ-12 had a three-factor structure and the GHQ-20 had a four-factor structure, which was superior to the GHQ-12 as it provided an additional factor named anhedonia suggesting some discriminative power (Penninkilampi-Kerola et al., 2006). The aforementioned studies that suggested a three-factor structure provided little information beyond that of a general factor while this study showed more information with the four-factor structure after SEM. The three-factor structure was similar to this four-factor structure with similar loadings but with differences regarding load on each factor. The main difference was observed in the factor orderings such that, the 3 items in Factor II (Anxiety and Depression) were the same as 3 of the 4 items in Factor II and 2 items in Factor I (Social Dysfunction) were similar to the 2 of 3 items in Factor III (Sánchez-López & Dresch, 2008).

The data were also explored for modification indices (MI) suggesting covariance between ‘Been losing confidence in yourself?’ and ‘Been thinking of yourself as a worthless person?’ (MI:40.87), ‘Been feeling unhappy and depressed?’ and ‘Been thinking of yourself as a worthless person?’ (MI:26.51), and ‘Felt constantly under strain?’ and ‘Been thinking of yourself as a worthless person?’ (MI:27.87) (Online Appendix 1, Supplementary files 2). This may be explained by the existence of an unspecified factor not included in the model that might partially account for the relationship between the variables or measurement artefacts. The existence of between-factor differences suggests that the GHQ-12 has multidimensional characteristics not captured by a severity score as reported by Vanheule and Bogaerts in a Belgian sample (Vanheule & Bogaerts, 2005) and Graetz in an Australian sample of young people (Graetz, 1991).

The GHQ-12 is a relatively brief tool, can easily be scored and has been previously used in different settings for screening purposes to detect psychological distress. The adequacy of the GHQ-12 was first tested and the assumptions for inferential analysis in South African HCWs were assessed. The EFA identified the factor structure and the CFA tested the theoretical foundation of the hypothesised factor structure further establishing the model fit.

Limitations of the study

The interpretation of the factors is complicated by the lack of prior knowledge leading to difficulty in interpretation of the results obtained from EFA. However, the use of CFA and SEM with standardised coefficients permitted the confirmation of the factors. The longer versions of the GHQ are useful in assessing the degree of psychological morbidity and outcomes for clients managed at mental services (Campbell et al., 2003). The primordial strength of this study is the validation of the utility of the GHQ-12 among South African HCWs. The limitation of this study includes the non-generalisability of the results to the whole South African and African population for screening as this study was limited to HCWs only. There may also be differences in the various professional disciplines such that what nurses may experience might differ from the experiences of medical doctors because of the nature of their roles.


Conclusion


The GHQ-12 displayed adequate reliability and validity in measuring psychological distress among South African HCWs. The factor structure suggested multidimensionality rather than a unidimensional construct. The findings of this study affirm the effectiveness of the GHQ-12 in a professional group of South Africans. The GHQ-12 can be a useful screening instrument in the South African population for general symptoms of mental distress to effectively assess overall psychological well-being and detect non-psychiatric challenges. Further research is warranted with larger sample sizes to test the reliability and validity of the GHQ-12 in the general South African population.



Acknowledgements


The authors are grateful to the healthcare workers who consented and participated in this study and the Gauteng Department of Health (GDOH) for the assistance.

This article accepted for publication by African Journal of Psychological Assessment (AJOPA) entitled ‘Reliability, validity and dimensionality of the GHQ-12 among South African Health Care Workers: Structural Equation Modelling’ was previously posted on medical archives website: Reliability, validity and dimensionality of the GHQ-12 in South African populations: Structural equation modelling (Kufe et al., n.d.) as a Preprint, https://doi.org/10.1101/2023.06.06.23290967.


Competing interests

The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.



Authors’ contributions

C.K.N. analysed the data and drafted and revised the manuscript. C.B. and K.W. conceived the study. C.K.N. was responsible for the overall content as the guarantor. All authors read and approved the final version of the manuscript.



Funding information

The study received funding from the National Institute for Occupational Health (NIOH), Johannesburg, South Africa and the University of British Columbia, Canada.



Data availability

Data supporting the findings of this study are available from the corresponding author, C.K.N., on request.



Disclaimer

The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of any affiliated agency of the authors.





References



Abubakar, A., & Fischer, R. (2012). The factor structure of the 12-item general health Questionnaire in a Literate Kenyan population. Stress and Health, 28(3), 248–254. https://doi.org/10.1002/smi.1420

Anjara, S.G., Bonetto, C., Van Bortel, T., & Brayne, C. (2020). Using the GHQ-12 to screen for mental health problems among primary care patients: Psychometrics and practical considerations. International Journal of Mental Health Systems, 14(1), 1–13. https://doi.org/10.1186/s13033-020-00397-0

Burvill, P.W., & Knuiman, M. (1983). Which version of the General Health Questionnaire should be used in community studies?. Australian and New Zealand Journal of Psychiatry, 17(3), 237–242. https://doi.org/10.3109/00048678309161279

Buselli, R., Corsi, M., Veltri, A., Baldanzi, S., Chiumiento, M., Lupo, E.D., Marino, R., Necciari, G., Caldi, F., Foddis, R., Guglielmi, G., & Cristaudo, A. (2021). Mental health of Health Care Workers (HCWs): A review of organizational interventions put in place by local institutions to cope with new psychosocial challenges resulting from COVID-19. Psychiatry Research, 299, 293. https://doi.org/10.1016/j.psychres.2021.113847

Campbell, A., Walker, J., & Farrell, G. (2003). Confirmatory factor analysis of the GHQ-12: Can I see that again?. Australian and New Zealand Journal of Psychiatry, 37(4), 475–483. https://doi.org/10.1046/j.1440-1614.2003.01208.x

Chen, Q., Liang, M., Li, Y., Guo, J., Fei, D., Wang, L., He, L., Sheng, C., Cai, Y., Li, X., Wang, J., & Zhang, Z. (2020). Mental health care for medical staff in China during the COVID-19 outbreak. The Lancet Psychiatry, 7(4), e15–e16. https://doi.org/10.1016/S2215-0366(20)30078-X

Corti, L. (1994). For better or worse? Annual change in smoking, self-assessed health and subjective wellbeing. In N. Buck, J. Gershuny, & S.J. Colchester 1990–1992 (eds.). University of Essex.

Donath, S. (2001). The validity of the 12-item General Health Questionnaire in Australia: A comparison between three scoring methods. Australian and New Zealand Journal of Psychiatry, 35(2), 231–235. https://doi.org/10.1046/j.1440-1614.2001.00869.x

Doyle, F., Watson, R., Morgan, K., & McBride, O. (2012). A hierarchy of distress and invariant item ordering in the General Health Questionnaire-12. Journal of Affective Disorders, 139(1), 85–88. https://doi.org/10.1016/j.jad.2011.10.022

Dutheil, F., Aubert, C., Pereira, B., Dambrun, M., Moustafa, F., Mermillod, M., Baker, J.S., Trousselard, M., Lesage, F.X., & Navel, V. (2019). Suicide among physicians and health-care workers: A systematic review and meta-analysis. PLoS One, 14(12), 1–28. https://doi.org/10.1371/journal.pone.0226361

El-Metwally, A., Javed, S., Razzak, H.A., Aldossari, K.K., Aldiab, A., Al-Ghamdi, S.H., Househ, M., Shubair, M.M., & Al-Zahrani, J.M. (2018). The factor structure of the general health questionnaire (GHQ12) in Saudi Arabia. BMC Health Services Research, 18(1), 1–11. https://doi.org/10.1186/s12913-018-3381-6

Fabrigar, L.R., Wegener, D.T., MacCallum, R.C., & Strahan, E. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272–299. https://doi.org/10.1037/1082-989X.4.3.272

Gelaye, B., Tadesse, M.G., Lohsoonthorn, V., Lertmeharit, S., Pensuksan, W.C., Sanchez, S.E., Lemma, S., Berhane, Y., Vélez, J.C., Barbosa, C., & Anderade, A.W.M. (2015). Psychometric properties and factor structure of the General Health Questionnaire as a screening tool for anxiety and depressive symptoms in a multi-national study of young adults. Journal of Affective Disorders, 15(187), 197–202. https://doi.org/10.1016/j.jad.2015.08.045

Goldberg, D. (1972). The detection of psychiatric illness by questionnaire. Oxford University Press.

Goldberg, D. (1989). Screening for psychiatric disorder The scope of epidemiological psychiatry: Essays in honour of Michael Shepherd (Williams P). Routledge.

Gorsuch, R. (1983). Factor analysis (2nd edn.). Lawrence Erlbaum Associates.

Graetz, B. (1991). Multidimensional properties of the General Health Questionnaire. Social Psychiatry and Psychiatric Epidemiology, 26(3), 132–138. https://doi.org/10.1007/BF00782952

Hair, J.F., Black, W.C., Babin, B.J., & Anderson, R. (2010). Multivariate data analysis (7th edn.). Pearson Prentice Hall.

Harris, P.A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J.G. (2009). Research Electronic Data Capture (REDCap) – A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 42(2), 377–381. https://doi.org/10.1016/j.jbi.2008.08.010

Henson, R.K., & Roberts, J. (2006). Use of Exploratory Factor Analysis in published research: Common errors and some comment on improved practice. Educational and Psychological Measurement, 66, 393–416. https://doi.org/10.1177/0013164405282485

Hooper, D., Coughlan, J., & Mullen, M.R. (2008). Structural equation modelling: Guidelines for determining model fit. Electronic Journal of Business Research Methods, 6(1), 53–60.

Hu, L-t., & Bentler, P. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

Hu, Y., Stewart-Brown, S., Twigg, L., & WeichEICH, S. (2007). Can the 12-item General Health Questionnaire be used to measure positive mental health?. Psychological Medicine, 37(7), 1005–1013. https://doi.org/10.1017/S0033291707009993

Hystad, S.W., & Johnsen, B.H. (2020). The dimensionality of the 12-item General Health Questionnaire (GHQ-12): Comparisons of factor structures and invariance across samples and time. Frontiers in Psychology, 11(June), 1–11. https://doi.org/10.3389/fpsyg.2020.01300

Illness, P., Goldberg, D.P., & Joubert, J. (1972). Adolescent separation anxiety is a hand – The detection of psychiatric illness by questionnaire ; A technique for the identification and assessment of non-psychotic. Canadian Psychiatric Association Journal, 18(4), 348–349. https://doi.org/10.1177/070674377301800421

Jawad, M.J., Abbas, M.M., Jawad, M.J., Hassan, S.M., & Hadi, N.R. (2021). Mental health and psychosocial considerations post Covid-19 outbreak. Wiadomosci Lekarskie (Warsaw, Poland : 1960), 74(12), 3156–3159. https://doi.org/10.36740/WLek202112106

Kang, L., Li, Y., Hu, S., Chen, M., Yang, C., Yang, B.X., Wang, Y., Hu, J., Lai, J., Ma, X., Chen, J., Guan, L., Wang, G., Ma, H., & Liu, Z. (2020). The mental health of medical workers in Wuhan, China dealing with the 2019 novel coronavirus. The Lancet Psychiatry, 7(3), e14. https://doi.org/10.1016/S2215-0366(20)30047-X

Kline, P. (2014). An easy guide to factor analysis. Guilford Press.

Kline, R. (2015). Principles and practice of Structural Equation Modeling (4th edn.). Guilford.

Knaak, S., Mantler, E., & Szeto, A. (2017). Mental illness-related stigma in healthcare: Barriers to access and care and evidence-based solutions. Healthcare Management Forum, 30(2), 111–116. https://doi.org/10.1177/0840470416679413

Kufe, C.N., Bernstein, C., & Wilson, K. (n.d.). Reliability, validity and dimensionality of the GHQ-12 in South African populations: Structural equation modelling (SEM). medRxiv. Preprint. https://doi.org/10.1101/2023.06.06.23290967

Lai, J., Ma, S., Wang, Y., Cai, Z., Hu, J., Wei, N., Wu, J., Du, H., Chen, T., Li, R., Tan, H., Kang, L., Yao, L., Huang, M., Wang, H., Wang, G., Liu, Z., & Hu, S. (2020). Factors associated with mental health outcomes among health care workers exposed to Coronavirus Disease 2019. JAMA Network Open, 2(3), e203976. https://doi.org/10.1001/jamanetworkopen.2020.3976

Lee, H.L., Wilson, K.S., Bernstein, C., Naicker, N., Yassi, A., & Spiegel, J.M. (2022). Psychological distress in South African healthcare workers early in the COVID-19 pandemic: An analysis of associations and mitigating factors. International Journal of Environmental Research and Public Health, 19(15), 9722. https://doi.org/10.3390/ijerph19159722

Liang, Y., Wang, L., & Yin, X. (2016). The factor structure of the 12-item general health questionnaire (GHQ-12) in young Chinese civil servants. Health and Quality of Life Outcomes, 14(1), 1–9. https://doi.org/10.1186/s12955-016-0539-y

Lloret, S., Ferreres, A., & Tomás, A.H.I. (2017). The exploratory factor analysis of items: Guided analysis based on empirical data and software. Anales de Psicologia, 33(2), 417–432. https://doi.org/10.6018/analesps.33.2.270211

Loehlin, J.C. (2017). Latent variables models. An introduction to factor, path and structural equation analysis. In J.C. Loehlin & A.A. Beaujean (Eds.), Latent variable models (5th ed., p. 390). Routledge.

Lundin, A., Hallgren, M., Theobald, H., & Hellgren, C.T.M. (2016). Validity of the 12-item version of the General Health Questionnaire in detecting depression in the general population. Public Health, 136, 66–74. https://doi.org/10.1016/j.puhe.2016.03.005

MacCallum, RC., Browne, M.W., & Sugawara, H. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1(2), 130–149. https://doi.org/10.1037/1082-989X.1.2.130

Mäkikangas, A., Feldt, T., Kinnunen, U., Tolvanen, A., Kinnunen, M.-L., & Pulkkinen, L. (2006). The factor structure and factorial invariance of the 12-item General Health Questionnaire (GHQ-12) across time: Evidence from two community-based samples. Psychological Assessment, 18(4), 444–451. https://doi.org/10.1037/1040-3590.18.4.444

Malkewitz, C.P., Schwall, P., Meesters, C., & Hardt, J. (2023). Estimating reliability: A comparison of Cronbach’s α, McDonald’s ωt and the greatest lower bound. Social Sciences & Humanities Open, 7(1), 100368. https://doi.org/10.1016/j.ssaho.2022.100368

McDonald, R.P., & Ho, M.-H. (2002). Principles and practice in reporting structural equation analyses. Psychological Methods, 7(1), 64–82. https://doi.org/10.1037/1082-989X.7.1.64

Mehta, S.S., Matthew, L., & Edwards, M.D. (2018). Suffering in silence: Mental health stigma and physicians’ licensing fears. American Journal of Psychiatry Residents’ Journal, 13(11), 2–4. https://doi.org/10.1176/appi.ajp-rj.2018.131101

Norris, M., & Lecavalier, L. (2010). Evaluating the use of exploratory factor analysis in developmental disability psychological research. Journal of Autism and Developmental Disorders, 40, 8–20. https://doi.org/10.1007/s10803-009-0816-2

Oliveira, T.A.A., Gouveia, V.V., Ribeiro, M.G.C., Oliveira, K.G., Melo, R.LP.D., & Montagna, E. (2023). General Health Questionnaire (GHQ12): New evidence of construct validity. Ciencia & Saude Coletiva, 28(3), 803–810. https://doi.org/10.1590/1413-81232023283.09452022

Özdemir, H., & Rezaki, M. (2007). General Health Questionnaire-12 for the detection of depression. Turk Psikiyatri Dergisi, 18(1), 1–8.

Padilla, M.A., & Divers, J. (2016). A comparison of composite reliability estimators: Coefficient Omega confidence intervals in the current literature. Educational and Psychological Measurement, 76(3), 436–453. https://doi.org/10.1177/0013164415593776

Penninkilampi-Kerola, V., Miettunen, J., & Ebeling, H. (2006). A comparative assessment of the factor structures and psychometric properties of the GHQ-12 and the GHQ-20 based on data from a Finnish population-based sample. Scandinavian Journal of Psychology, 47(5), 431–440. https://doi.org/10.1111/j.1467-9450.2006.00551.x

Rana, W., Mukhtar, S., & Mukhtar, S. (2020). Mental health of medical workers in Pakistan during the pandemic COVID-19 outbreak. Asian Journal of Psychiatry, 51, 102080). https://doi.org/10.1016/j.ajp.2020.102080

Reymont, R.A., & Jvreskog, K.G. (1993). Applied factor analysis in the natural sciences. Cambridge University Press.

Russell, D.W. (2002). In search of underlying dimensions: The use (and abuse) of factor analysis in personality and social psychology bulletin. Personality and Social Psychology Bulletin, 28(12), 1629–1646. https://doi.org/10.1177/014616702237645

Sánchez-López, M.D.P., & Dresch, V. (2008). The 12-item General Health Questionnaire (GHQ-12): Reliability, external validity and factor structure in the Spanish population. Psicothema, 20(4), 839–843.

Shaukat, N., Mansoor, A., & Razzak, J. (2020). Physical and mental health impacts of COVID-19 on healthcare workers: A scoping review. International Journal of Emergency Medicine, 13(1), 1–8. https://doi.org/10.1186/s12245-020-00299-5

Shevlin, M., & Adamson, G. (2005). Alternative factor models and factorial invariance of the GHQ-12: A large sample analysis using confirmatory factor analysis. Psychological Assessment, 17(2), 231–236. https://doi.org/10.1037/1040-3590.17.2.231

Skogen, J.C., Øverland, S., Smith, O.R.F., & Leif, E.A. (2017). The factor structure of the Hopkins Symptoms Checklist (HSCL-25) in a student population: A cautionary tale. Scandinavian Journal of Public Health, 45(4), 357–365. https://doi.org/10.1177/1403494817700287

Søvold, L.E., Naslund, J.A., Kousoulis, A.A., Saxena, S., Qoronfleh, M.W., Grobler, C., & Münter, L. (2021). Prioritizing the mental health and well-being of healthcare workers: An urgent global public health priority. Frontiers in Public Health, 9, 1–12. https://doi.org/10.3389/fpubh.2021.679397

Uras, C., Delle Poggi, A., Rocco, G., & Tabolli, S. (2012). General Health Questionnaire-12 [Psychological wellbeing and risk of anxiety/depression in nursing students measured with the General Health Questionnaire-12]. Assistenza infermieristica e ricerca, 31(2), 70–75. https://doi.org/10.1702/1131.12468

Vanheule, S., & Bogaerts, S. (2005). Short communication: The factorial structure of the GHQ-12. Stress and Health: Journal of the International Society for the Investigation of Stress, 21(4), 217–222. https://doi.org/10.1002/smi.1058

Vizheh, M., Qorbani, M., Arzaghi, S.M., Muhidin, S., Javanmard, Z., & Esmaeili, M. (2020). The mental health of healthcare workers in the COVID-19 pandemic: A systematic review. Journal of Diabetes and Metabolic Disorders, 19(2), 1967–1978. https://doi.org/10.1007/s40200-020-00643-9

Watkins, M.W. (2018). Exploratory Factor Analysis: A guide to best practice. Journal of Black Psychology, 44(3), 219–246. https://doi.org/10.1177/0095798418771807

Werneke, U., Goldberg, D.P., Yalcin, I., & Üstün, B. (2000). The stability of the factor structure of the General Health Questionnaire. Psychological Medicine, 30(4), 823–829. https://doi.org/10.1017/S0033291799002287

WHO. (2020). Mental health and psychosocial considerations during the COVID-19 outbreak. Who Guideline.

Young, A.G., & Pearce, S. (2013). A beginner’s guide to factor analaysis: Focusing on Exploratory Factor Analysis. Totorials in Quantitative Methods for Psychology, 9(2), 79–94. https://doi.org/10.20982/TQMP.09.2.P079










