Assessing Diagnostic Precision: Adaptations of the Hopkins Symptom Checklist (HSCL-5/10/25) Among Tertiary-Level Students in Norway Research Articles Assessing Diagnostic Precision: Adaptations of the Hopkins Symptom Checklist (HSCL-5/10/25) Among Tertiary-Level Students in Norway Børge Sivertsen 1,2 , Jens C. Skogen 1,3,4 , Anne Reneflot 5 , Marit Knapstad 1 , Otto Robert Frans Smith 1,4,6 , Leif Edvard Aarø 1 , Benedicte Kirkøen 4 , Bengt Oscar Lagerstrøm 7, Ann Kristin Skrindo Knudsen 8 [1] Department of Health Promotion, Norwegian Institute of Public Health, Bergen, Norway. [2] Department of Research & Innovation, Helse-Fonna HF, Haugesund, Norway. [3] Center for Alcohol & Drug Research, Stavanger University Hospital, Stavanger, Norway. [4] Centre for Evaluation of Public Health Measures, Norwegian Institute of Public Health, Oslo, Norway. [5] Department of Mental Health and Suicide, Norwegian Institute of Public Health, Oslo, Norway. [6] Department of Teacher Education, NLA University College, Bergen, Norway. [7] Department for Methodology and Data Collection, Statistics Norway, Oslo, Norway. [8] Department of Disease Burden, Norwegian Institute of Public Health, Bergen, Norway. Clinical Psychology in Europe, 2024, Vol. 6(4), Article e13275, https://doi.org/10.32872/cpe.13275 Received: 2023-11-27 • Accepted: 2024-08-01 • Published (VoR): 2024-12-20 Handling Editor: Winfried Rief, Philipps-University of Marburg, Marburg, Germany Corresponding Author: Børge Sivertsen, Department of Health Promotion, Norwegian Institute of Public Health, Zander Kaaes gate 7, 5018 Bergen, Norway. Phone: +47 99 76 92 62. E-mail: borge.sivertsen@fhi.no Abstract Background: Universities worldwide are witnessing a surge in mental health problems among students, particularly in anxiety and depression. The Hopkins Symptom Checklist (HSCL) is a popular screening tool, but its reliability in identifying mental disorders remains debated. The aim of this study was to evaluate the criterion validity of the HSCL-25, HSCL-10, and HSCL-5 using 30- day prevalence of major depressive episode (MDE) and generalized anxiety disorder (GAD) from a self-administered electronic version of the Composite International Diagnostic Interview, fifth version (CIDI 5.0), as the benchmark. Method: Data stem from a national survey targeting students in higher education in Norway. In a 2023 follow-up study on mental disorders, 5,568 participants completed both the HSCL-25 and the CIDI. Sex-specific optimal thresholds for all HSCL versions in relation to MDE and GAD (from CIDI) were determined using the Youden Index maximization. This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International License, CC BY 4.0, which permits unrestricted use, distribution, and reproduction, provided the original work is properly cited. https://crossmark.crossref.org/dialog/?doi=10.32872/cpe.13275&domain=pdf&date_stamp=2024-12-20 https://orcid.org/0000-0003-4654-9296 https://orcid.org/0000-0003-0722-5440 https://orcid.org/0000-0003-0536-5271 https://orcid.org/0000-0003-0958-1596 https://orcid.org/0000-0002-6306-2239 https://orcid.org/0000-0001-5027-3611 https://orcid.org/0000-0003-2641-5477 https://orcid.org/0000-0002-1218-798X https://www.psychopen.eu/ https://cpe.psychopen.eu/ https://creativecommons.org/licenses/by/4.0/ https://creativecommons.org/licenses/by/4.0/ https://creativecommons.org/licenses/by/4.0/ Results: The optimal cut-off values for detecting MDE or GAD with the HSCL-25 were 1.96 for males and 2.20 for females, displaying a good balance between sensitivity and specificity. Similar high and balanced sensitivity and specificity patterns were found for both the HSCL-10 and HSCL-5. However, all HSCL versions overestimated prevalence rates compared to the self- administered CIDI. Conclusions: All three HSCL versions showed high criterion validity. The data indicate that HSCL may be better as a screening tool than for precise estimation of MDE and GAD prevalence. For improved diagnostic accuracy, future HSCL versions should incorporate functional impairment assessment. This update would bring the HSCL into closer alignment with clinical diagnostic standards. Keywords depression, anxiety, students, young adults, questionnaires, psychometrics Highlights • This study assessed HSCL's ability to detect anxiety and depression in Norwegian college students. • A self-administered version of CIDI was used as the gold standard for diagnostic accuracy. • All three HSCL versions showed high criterion validity and good diagnostic precision. • HSCL may be better as a screening tool than for precise estimation of mental disorders. Universities and colleges worldwide are confronting a concerning rise in the incidence of mental health issues among their students, with projections significantly surpassing those observed in the general population (Ibrahim et al., 2013; McCloud et al., 2023). Recent systematic reviews on anxiety and depression have yielded a cumulative annual prevalence range of 25-30% among tertiary education students (Chi et al., 2023; Ibrahim et al., 2013; Kou et al., 2012; Sheldon et al., 2021). Although diagnostic interviews maintain their status as the benchmark for mental disorder diagnosis (Nordgaard et al., 2013; Rettew et al., 2009), their demanding time and resource requirements hinder most researchers from incorporating comprehensive psychiatric interviews into their assessment battery. Consequently, a substantial portion of investigations are constrained to incorporating brief survey questionnaires when evaluating mental health issues (Auerbach et al., 2016). However, in order to obtain prev­ alence estimates that are as accurate as possible, it is important that the case-detection capabilities of these briefer questionnaires are thoroughly evaluated. Also, considering the changing trends and disparities observed across age cohorts and research popula­ tions, it is imperative to re-examine previously established cut-off values to ensure that the case-detection capabilities are effectively validated for each distinct study population. HSCL's Diagnostic Precision in Tertiary Students 2 Clinical Psychology in Europe 2024, Vol. 6(4), Article e13275 https://doi.org/10.32872/cpe.13275 https://www.psychopen.eu/ In this context, a frequently employed survey instrument is the Hopkins Symptom Checklist (HSCL), initially devised during the 1950s as a clinical tool for assessing symptoms of several mental disorders. Although the original iteration encompassed an extensive array of mental disorders, one of the most commonly used versions today is the HSCL-25. This abbreviated version focuses on two symptom dimensions, anxiety and depression. However, the capacity of the HSCL-25 to differentiate between these two conditions across different sexes is not fully established (Sandanger et al., 1998; Skogen et al., 2017). Since the 1990s, the efficacy of HSCL-25 in identifying cases has been examined a few times through comparisons with structured diagnostic interviews, suggesting that while HSCL-25 performs adequately in detecting depression, the results for anxiety are more variable (Sandanger et al., 1998; Veijola et al., 2003). The HSCL-25 uses a scoring range of 1 to 4, and to determine the commonly used mean HSCL-25 score, the total score is divided by the item count. A traditional cut-off value of 1.75 is commonly employed to indicate major depressive disorder (Glaesmer et al., 2014; Sandanger et al., 1998; Veijola et al., 2003). However, a recent Spanish study has challenged this one-size-fits-all approach, proposing distinct optimal cut-off values for women (1.76) and men (1.84) (Rodríguez-Barragán et al., 2021). This suggests that sex-specific thresholds might be necessary for more accurate diagnosis, considering the different ways in which men and women experience and report mental health symptoms. Further abbreviated iterations of the HSCL have been developed, with both HSCL-5 and HSCL-10 now in widespread use. However, few studies have investigated the case- detection capability of these versions, and none have done so for males and females sepa­ rately. To the best of our knowledge, just one study has directly contrasted HSCL-5 and HSCL-10 against a structured diagnostic interview. A recent Spanish study demonstrated good reliability and validity of both tools for detecting depression (Rodríguez-Barragán et al., 2023). Additionally, a recent Norwegian study assessed HSCL-5's capacity for iden­ tifying cases in the general population, and its findings indicate that HSCL-5 effectively identifies people with generalized anxiety disorder or major depressive disorder within this context (Kirkøen et al., Manuscript in preparation). Based on these considerations, the objective of this current study is to assess the proficiency of HSCL-25, HSCL-10, and HSCL-5 in detecting cases of major depressive episode (MDE) and generalized anxiety disorder (GAD) in a national sample of college and university students. Our focus is on potential sex-specific cut-off values and differ­ ences, using a recently developed self-administered electronic version of the Composite International Diagnostic Interview (CIDI) 5.0. This approach aims to enhance the accura­ cy of case-detection and ensure that the nuanced mental health experiences of both sexes are adequately captured. Sivertsen, Skogen, Reneflot et al. 3 Clinical Psychology in Europe 2024, Vol. 6(4), Article e13275 https://doi.org/10.32872/cpe.13275 https://www.psychopen.eu/ Method Setting and Participants The primary population for this study is derived from the Norwegian SHOT study (Students' Health and Wellbeing Study), a nationwide survey that centres on students pursuing higher education. Since 2010, four major surveys have been conducted, with the latest wave conducted in 2022. The SHOT2022 survey comprehensively explored a multitude of dimensions encompassing health and lifestyle. These dimensions included psychological distress, suicidality, life satisfaction, loneliness, sleep problems, sexual harassment, pain, physical exercise, alcohol and drug use, as well as demographic and educational parameters. Comprehensive information concerning the SHOT study has been previously documented (Sivertsen et al., 2019). During the survey period, SHOT2022 was distributed electronically via a web-based platform and was open for submissions from February 8 to April 19, 2022. Invitations for participation were sent to all full-time Norwegian students engaged in higher education, both within the country and abroad. Extensive efforts were undertaken to increase awareness about the study through channels such as email, SMS, and informational campaigns conducted by welfare organizations and educational institutions. A total of 169,572 students met the study's inclusion criteria, which required them to be full-time college or university students and hold Norwegian citizenship. Of these, 59,544 students completed the online questionnaire after receiving two reminders. This resulted in a re­ sponse rate of 35.1% (which did not differ between geographical regions). For the present study, the inclusion criteria specified that participants be between the ages of 18 and 35 years. Consequently, a subset of 53,362 students within this age range was selected for analysis. When consenting to participate in the SHOT2022, students were given the option to express their interest in participating in a follow-up study on mental disorders. Out of the total participants, 26,311 students consented to be a part of this follow-up study. To better reflect the sex distribution of the base study population, more male students were invited to participate in the CIDI study. Consequently, 16,418 students, officially registered as of January 2023, were invited. However, fewer male students consented to follow-up contact compared to females. This led to females comprising a higher proportion (70.4%) of the invitations for the CIDI study. Figure 1 illustrates the participation process for the current study. Out of 9,552 students who provided valid responses on both the HSCL-25 and at least one of the CIDI diagnostic sections, half were randomly selected to complete the HSCL-25 before the CIDI. This resulted in a subset of 5,076 participants who provided valid scores on the HSCL-25 and then completed the CIDI. The remaining half completed the assessments in the reverse order. This deliberate sequencing was chosen to facilitate future investiga­ tions into the potential influences of the order of questionnaire administration on the reported results from the HSCL-25. HSCL's Diagnostic Precision in Tertiary Students 4 Clinical Psychology in Europe 2024, Vol. 6(4), Article e13275 https://doi.org/10.32872/cpe.13275 https://www.psychopen.eu/ Figure 1 Participant Flow in the Study Study population: All fulltime students in Norway aged 18-35 years per January 2022 N = 164,716 SHOT 2022 sample Students aged 18-35 years n = 53,362 Invited to CIDI n = 16,418 CIDI responders n = 9,552 ------------------------------ Valid response on both HSCL-25 and at least one diagnostic section Not invited to CIDI n = 36,817 ------------------------------ Women removed after sampling procedure n = 7,070 Did not consent to be contacted for FU study n = 29,747 Non-response n = 111,354 CIDI non-responders n = 5,435 ------------------------------ Excluded from sample n = 523 No valid diagnostic sections = 522 Age outside age range n = 1 Included sample n = 5,076 Males = 1,493 Females = 3,583 Randomized to complete CIDI after HSCL Excluded Randomized to complete CIDI before HSCL Sivertsen, Skogen, Reneflot et al. 5 Clinical Psychology in Europe 2024, Vol. 6(4), Article e13275 https://doi.org/10.32872/cpe.13275 https://www.psychopen.eu/ For the purposes of the present study, we only used observations where the HSCL-25 was administered before the CIDI, as screening instruments are normally not preceded by full diagnostic assessments in population-based surveys. The CIDI study took place between January 24 and February 6, 2023, approximately 12 months after the SHOT2022 survey was conducted. More detailed information on the participation process has been published elsewhere (Sivertsen et al., 2023). Instruments Sociodemographic Information Participants' age and sex information were derived from their 11-digit Norwegian nation­ al identity numbers. Depression and Anxiety Disorders: The CIDI The data collection utilized a recently developed electronic, self-administered version of the Composite International Diagnostic Interview, fifth version (CIDI 5.0), developed for the World Health Organization (WHO) World Mental Health (WMH) Surveys (Kessler & Üstün, 2004). Both the original CIDI and this self-administered version maintained the same sequence of CIDI modules, and the wording of the questions remained nearly identical, though instructions were slightly adapted for self-administration. To reduce participant burden and enhance response rates, the current version excluded certain diagnostic categories, namely bipolar, obsessive-compulsive, PTSD, ADHD, and person­ ality disorders. In short, CIDI 5.0 is a standardized interview that assesses 30-days, 12 months and lifetime prevalence for several mental and substance use disorders according to diagnostic criteria in the Diagnostic and Statistical Manual of Mental Disorders 5th edition (DSM-5) (American Psychiatric Association, 2013). The interview version of CIDI 3.0 has shown good concordance with diagnostic instruments such as the Structured Clinical Interview for DSM-IV (SCID) (Haro et al., 2006) and Schedules for Clinical Assessment in Neuropsychiatry (SCAN) (Jordanova et al., 2004). Current mental disorder was defined as the presence of MDE or GAD during the 30 days before study participation. We also estimated prevalence rates for mental disorders spanning a 12-month period and over the lifetime, but these estimations were not incor­ porated into the present study, aligning with its specific emphasis on assessing current mental disorders in comparison with the HSCL. The operationalization of these diagno­ ses was based on algorithms developed for CIDI 5.0 in the WMH Surveys Initiative. The Hopkins Symptom Checklist (HSCL) Mental health problems in the 14 days before the survey were assessed by the widely used Hopkins Symptom Checklist (HSCL-25) (Derogatis et al., 1974), derived from the 90-item Symptom Checklist (SCL-90). The score for the HSCL is determined by taking HSCL's Diagnostic Precision in Tertiary Students 6 Clinical Psychology in Europe 2024, Vol. 6(4), Article e13275 https://doi.org/10.32872/cpe.13275 https://www.psychopen.eu/ the sum of the item scores and dividing it by the total number of items responded to, yielding a potential range of 1 to 4. An investigation of the factor structure based on the SHOT2014 dataset showed that a unidimensional model had the best psychometric prop­ erties in the student population and not the original subscales of anxiety and depression (Skogen et al., 2017). Details on the development of mental health problems assessed with the HSCL-25 in the SHOT waves were recently published by Knapstad and colleagues (Knapstad et al., 2021). The items included in the three different versions of the HSCL are listed in Table 1. Table 1 Items Included in the Different HSCL Iterationsa Item content HSCL-25 HSCL-10 HSCL-5 1. Suddenly scared for no reason ◯ ◯ 2. Feeling fearful ◯ ◯ ◯ 3. Faintness, dizziness, or weakness ◯ ◯ 4. Nervousness or shakiness inside ◯ ◯ 5. Heart pounding or racing ◯ 6. Trembling ◯ 7. Feeling tense or keyed up ◯ ◯ 8. Headaches ◯ 9. Spells of terror or panic ◯ 10. Feeling restless, can't sit still ◯ 11. Feeling low in energy, slowed down ◯ 12. Blaming yourself for things ◯ ◯ 13. Crying easily ◯ 14. Loss of sexual interest or pleasure ◯ 15. Poor appetite ◯ 16. Difficulty falling asleep, staying asleep ◯ ◯ 17. Feeling hopeless about the future ◯ ◯ ◯ 18. Feeling blue ◯ ◯ ◯ 19. Feeling lonely ◯ 20. Feeling trapped or caught ◯ 21. Worrying too much about things ◯ ◯ 22. Feeling no interest in things ◯ 23. Thoughts of ending your life ◯ 24. Feeling everything is an effort ◯ ◯ 25. Feelings of worthlessness ◯ ◯ aEach symptom is rated on a four-point frequency scale (1 =  not at all, 2 = a little, 3  =  quite a lot, 4  =   extremely). Sivertsen, Skogen, Reneflot et al. 7 Clinical Psychology in Europe 2024, Vol. 6(4), Article e13275 https://doi.org/10.32872/cpe.13275 https://www.psychopen.eu/ Statistical Analyses In the present study, we first present summary statistics for HSCL scores and the prev­ alence of MDE and GAD, stratified by sex (Table 3 and Table 4). Subsequently, we proceeded to determine sex-specific optimal cut-off points for HSCL-25, HSCL-10, and HSCL-5 in relation to either MDE or GAD combined, and for MDE and GAD separately, as identified through CIDI. This estimation was grounded in the maximization of the Youden Index, which is a commonly used metric for binary classification in validation studies aimed at striking a balance between sensitivity and specificity. The formula for the Youden Index, denoted as ('sensitivity' + 'specificity') - 1, yields a scale ranging from 0 to 1. Higher values signify better discriminative capacity, where 0 denotes no discrimi­ nation, and 1 reflects perfect discrimination. Although rules of thumb always must be considered in conjunction with other aspects, a score below 0.5 on the Youden Index indicates that the test in question may not be useful as a classification tool, whereas a score surpassing 0.5 can be construed as indicating a valuable test. Alongside the Youden Index, we also present a comprehensive view of overall accuracy, sensitivity, specificity, positive and negative predictive values (PPV and NPV), and the area under the curve (AUC). All analyses were performed using R 4.2.2. Results As detailed in Table 2, the sample for the CIDI study primarily consisted of female students of Norwegian ethnicity, with an average age of 24 years, and about half were single. Most participants had parents with high educational levels. Compared with the overall SHOT2022 study, the sociodemographic characteristics were similar, except for a slightly higher proportion of females in the CIDI study (70%) compared to the SHOT2022 study (66%), as shown in Table 2. Non-respondents to the CIDI study, who were invited but did not participate, differed mainly in having parents with lower education levels. The level of mental health problems, measured by the HSCL-25 in the SHOT2022 study, was marginally lower in CIDI respondents (M = 1.88, SD = 0.61) compared to non-respondents (M = 1.90, SD = 0.61, Cohen’s d = 0.03). However, CIDI respondents had a slightly higher HSCL-25 score than the overall SHOT2022 sample (M = 1.86, SD = 0.57, Cohen’s d = 0.03). Figure 1 details the participation process of the current study. A total of 5,076 partic­ ipants completed both the CIDI and HSCL-25 in advance. The mental health characteris­ tics of the sample are detailed in Table 3. Females reported considerably higher average scores on all HSCL iterations, and the prevalences of 30-day MDE and GAD were much higher among females as well, as detailed in Table 3. HSCL's Diagnostic Precision in Tertiary Students 8 Clinical Psychology in Europe 2024, Vol. 6(4), Article e13275 https://doi.org/10.32872/cpe.13275 https://www.psychopen.eu/ Table 2 Demographical Characteristics and Representativeness of the CIDI Responders, CIDI Non-Responders and the Overall Sample Based on Data From 2022 Characteristic CIDI responders (n = 9,552) CIDI non-responders (n = 6,993) pa SHOT2022b (n = 53,362) pa Age, mean (SD) 24.03 (3.28) 23.97 (3.24) .24 23.98 (1.85) .14 Sex, % (n) .35 < .001 Women 70.0 (6,686) 71.0 (4,968) 66.4 (35,423) Men 30.0 (2,866) 29.0 (2,025) 33.6 (17,939) Marital status, % (n) .20 .81 Single 51.3 (4,904) 50.4 (3,526) 51.0 (27,197) Boy-/girlfriend 22.5 (2,152) 23.7 (1,659) 22.8 (12,152) Cohabitant 22.6 (2,156) 22.4 (1,563) 22.6 (12,058) Married/registered partner 3.2 (308) 3.0 (207) 3.1 (1,667) Missing 0.3 (32) 0.5 (38) 0.5 (288) Maternal education, % (n) .01 .27 Primary 4.3 (407) 5.3 (369) 4.5 (2,407) Secondary 27.2 (2,601) 27.6 (1,931) 27.6 (14,707) College/university 65.9 (6,290) 64.2 (4,488) 64.3 (34,326) Missing 2.7 (254) 2.9 (205) 3.6 (1,992) Paternal education, % (n) .02 .39 Primary 5.7 (544) 6.8 (473) 6.0 (3,182) Secondary 34.9 (3,335) 35.0 (2,449) 35.1 (18,735) College/university 54.6 (5,211) 52.7 (3,687) 53.3 (28,446) Missing 4.8 (462) 5.5 (384) 5.6 (2,999) HSCL-25, Mean (SD) 1.88 (0.61) 1.90 (0.61) .03 1.86 (0.59) < .001 Missing, % (n) 0.2 (17) 0.3 (24) 0.4 (214) Note. SHOT2022 = Students’ Health and Wellbeing Study 2022; CIDI = Composite International Diagnostic Interview; HSCL-25 = Hopkins Symptoms Checklist – 25 items version. acompared with the CIDI responders group (p-values based on Chi-squared test [categorical variables] or t-test [continuous variables]). bgrand mean for the SHOT2022 sample aged 18-35. Table 3 Mental Health Characteristics of the Study Sample Characteristic Males, N = 1,493 Females, N = 3,583 p HSCL-25, M (SD) 1.67 (0.53) 1.96 (0.60) < .001 HSCL-10, M (SD) 1.70 (0.59) 1.98 (0.67) < .001 HSCL-5, M (SD) 1.87 (0.74) 2.23 (0.80) < .001 Major depressive episode (MDE) 9.8% 16.9% < .001 Generalized anxiety disorder (GAD) 8.0% 15.7% < .001 MDE or GAD 13.1% 23.8% < .001 Sivertsen, Skogen, Reneflot et al. 9 Clinical Psychology in Europe 2024, Vol. 6(4), Article e13275 https://doi.org/10.32872/cpe.13275 https://www.psychopen.eu/ For the HSCL-25, the optimal cut-off values for identifying cases of MDE or GAD were 1.96 for males and 2.20 for females. There was a good balance between sensitivity (0.92 for males and 0.89 for females) and specificity (0.83 for both males and females), and the Youden Index was acceptable for both males (0.74) and females (0.72). The PPV and NPV for males were 0.45 and 0.99, while the corresponding numbers were 0.62 and 0.96 for females (see Table 4 and Table 5). For the HSCL-10, the best cut-off values to identify cases of MDE or GAD were 2.10 for males and 2.30 for females. The balance between sensitivity (0.91 for males and 0.88 for females) and specificity (0.81 for both sexes) was good. The Youden Index was satisfactory for both males (0.72) and females (0.69). The PPV and NPV for males registered at 0.42 and 0.98, respectively, and for females, these values were 0.58 and 0.96. For the HSCL-5, the optimal cut-off values to identify cases of MDE or GAD were 2.25 for males and 2.75 for females, and similar to the two longer HSCL iterations, the balance between sensitivity and specificity was notably good (see Tables 4 and 5 for details). The Youden Index showed satisfactory results for both males and females at 0.68, and the PPV and NPV for males stood at 0.41 and 0.97, respectively, whereas for females, these figures were 0.65 and 0.94. Table 4 Optimal Cut-Off Values for Females for HSCL-25, HSCL-10, and HSCL-5 for Major Depressive Episode (MDE) and Generalized Anxiety Disorder (GAD) Assessed by CIDI HSCL version and CIDI diagnosis Cut-off value Youden Index Accuracy Sensitivity Specificity PPV NPV AUC HSCL-25 MDE or GAD ≥ 2.20 0.72 0.85 0.89 0.83 0.62 0.96 0.94 MDE ≥ 2.28 0.74 0.84 0.92 0.82 0.51 0.98 0.94 GAD ≥ 2.08 0.64 0.76 0.91 0.73 0.38 0.98 0.90 HSCL-10 MDE or GAD ≥ 2.30 0.70 0.83 0.88 0.81 0.60 0.96 0.93 MDE ≥ 2.44 0.73 0.86 0.88 0.85 0.55 0.97 0.93 GAD ≥ 2.30 0.63 0.77 0.88 0.75 0.39 0.97 0.89 HSCL-5 MDE or GAD ≥ 2.75 0.68 0.86 0.81 0.87 0.67 0.94 0.91 MDE ≥ 2.75 0.69 0.83 0.86 0.83 0.50 0.97 0.91 GAD ≥ 2.80 0.64 0.82 0.83 0.81 0.45 0.96 0.89 Note. CIDI = Composite International Diagnostic Interview; HSCL = Hopkins Symptom Checklist; MDE = Major Depressive Episode; GAD = Generalized Anxiety Disorder; PPV = Positive Predictive Value; NPV = Negative Predictive Value; AUC = Area Under the ROC Curve. HSCL's Diagnostic Precision in Tertiary Students 10 Clinical Psychology in Europe 2024, Vol. 6(4), Article e13275 https://doi.org/10.32872/cpe.13275 https://www.psychopen.eu/ The corresponding values for only MDE and only GAD were relatively similar to those of MDE or GAD. The same optimal cut-off values for all HSCL iterations were replicated in bootstrapped analyses with 1,000 runs. As also detailed in Table 6, when using the optimal cut-offs, all three HSCL iterations were associated with a marked overestimation of the prevalences, according to the self-administered CIDI. For example, the 30-day prevalence rates of MDE or GAD were 13.1% for males and 23.8% for females. However, using the optimal HSCL-25 cut-offs, these rates increased to 26.9% for males and 33.8% for females, respectively. Discussion The present large-scale study of students in higher education employed a recently adap­ ted self-administered psychiatric diagnostic survey (CIDI 5.0) to investigate the efficacy of three iterations of the widely used HSCL scale in detecting cases of GAD or MDE, with an emphasis on potential sex-specific cut-off values. Our results show that all three versions of the HSCL discern relatively well between students afflicted with and without generalized anxiety disorder (GAD) or major depressive episode (MDE), and that different cut-offs for males and females should be used to ensure a good balance between sensitivity, specificity and overall accuracy. Table 5 Optimal Cut-Off Values for Males for HSCL-25, HSCL-10, and HSCL-5 for Major Depressive Episode (MDE) and Generalized Anxiety Disorder (GAD) Assessed by CIDI HSCL version and CIDI diagnosis Cut-off value Youden Index Accuracy Sensitivity Specificity PPV NPV AUC HSCL-25 MDE or GAD ≥ 1.96 0.74 0.84 0.92 0.83 0.45 0.99 0.94 MDE ≥ 2.00 0.76 0.84 0.94 0.82 0.37 0.99 0.95 GAD ≥ 1.96 0.69 0.80 0.91 0.79 0.27 0.99 0.92 HSCL-10 MDE or GAD ≥ 2.10 0.72 0.82 0.91 0.81 0.42 0.98 0.94 MDE ≥ 2.30 0.74 0.87 0.87 0.87 0.41 0.98 0.95 GAD ≥ 2.30 0.68 0.85 0.83 0.85 0.32 0.98 0.91 HSCL-5 MDE or GAD ≥ 2.25 0.68 0.83 0.85 0.82 0.42 0.97 0.92 MDE ≥ 2.25 0.68 0.81 0.88 0.80 0.32 0.98 0.92 GAD ≥ 2.20 0.64 0.74 0.92 0.72 0.22 0.99 0.91 Note. CIDI = Composite International Diagnostic Interview; HSCL = Hopkins Symptom Checklist; MDE = Major Depressive Episode; GAD = Generalized Anxiety Disorder; PPV = Positive Predictive Value; NPV = Negative Predictive Value; AUC = Area Under the ROC Curve. Sivertsen, Skogen, Reneflot et al. 11 Clinical Psychology in Europe 2024, Vol. 6(4), Article e13275 https://doi.org/10.32872/cpe.13275 https://www.psychopen.eu/ A significant finding in this study is the introduction of new, sex-specific cut-off values for all HSCL iterations. These values diverge from those in earlier validation studies, which might have lacked the statistical power to evaluate distinct cut-offs for males and females. While the conventional cut-off value of 1.75 for the HSCL-25 has been consistently used for both sexes for years, the present study indicates that adopting slightly elevated cut-offs enhances the balance between sensitivity and specificity for both males and females across all HSCL iterations. Prior research contrasting the full HSCL-25 with structured diagnostic interviews have demonstrated a sensitivity ranging from 70-88% and a specificity between 77-85% for mood disorders or depression, and 43-50% sensitivity with 83% specificity for anxiety disorders (Rodríguez-Barragán et al., 2021; Sandanger et al., 1998; Veijola et al., 2003). In the current study, both the sensitivity and specificity were generally a little higher both for MDE (sensitivity 92-94%, and specificity 82%), and GAD (sensitivity 91%, and specificity 73-79%). It is important to note that the different studies employ different anxiety and depression diagnoses. What is noteworthy is that both the shorter HSCL-10 and HSCL-5 displayed similarly high levels of sensitivity and specificity. Although very few studies have investigated the case-detection ability of these shorter HSCL iterations, a recent study by Rodríguez-Bar­ ragán et al. found that the HSCL-5 yielded a sensitivity of 78% and a specificity of 73% for depression. Based on our dataset using the newly suggested cut-offs, we observed an even higher sensitivity and specificity for both MDE and GAD, suggesting that both Table 6 Prevalence of Mental Disorder According to CIDI and the Optimal HSCL Cutoffs HSCL version and CIDI diagnosis Males Females CIDI HSCLa CIDI HSCLa HSCL-25 MDE or GAD 13.1% 26.9% 23.8% 33.8% MDE 9.8% 25.1% 16.9% 30.2% GAD 8.0% 26.9% 15.7% 39.3% HSCL-10 MDE or GAD 13.1% 28.3% 23.8% 35.2% MDE 9.8% 20.6% 16.9% 27.2% GAD 8.0% 20.6% 15.7% 35.2% HSCL-5 MDE or GAD 13.1% 26.6% 23.8% 28.7% MDE 9.8% 26.6% 16.9% 28.7% GAD 8.0% 33.3% 15.7% 28.6% Note. CIDI = Composite International Diagnostic Interview; HSCL = Hopkins Symptom Checklist; MDE = Major Depressive Episode; GAD = Generalized Anxiety Disorder. aEstimated prevalences based on the optimal HSCL cut-off. HSCL's Diagnostic Precision in Tertiary Students 12 Clinical Psychology in Europe 2024, Vol. 6(4), Article e13275 https://doi.org/10.32872/cpe.13275 https://www.psychopen.eu/ the HSCL-10 the HSCL-5 may be equally good alternatives to be used in epidemiological research for the purpose of detecting probable cases of depression and anxiety. However, it should be noted that the relative differences between HSCL and CIDI prevalences were more pronounced in men. This was driven by differences in base prevalence as measured by CIDI; specifically, a relatively lower CIDI prevalence estimate is, all other things being equal (ceteris paribus), associated with a higher HSCL prevalence estimate. Another explanation for the disparities between HSCL and CIDI prevalence is that HSCL does not assess how symptoms affect daily functioning, while this is a key requirement for diagnosing a mental disorder in the CIDI instrument. This difference might partly explain why specificity was somewhat lower for GAD than for MDE. Anxiety disorders often have a more intricate relationship with functional impairment compared to depression (McKnight et al., 2016). Future studies could explore improving the HSCL by adding a measure of daily functioning. This addition could address the tool's current limitation of potentially overestimating mental health disorders due to the lack of assessment of the real-world functional impact. A revised HSCL with functional impairment questions could offer a more thorough evaluation. Alternatively, it could be used initially for screening, followed by more detailed assessments of daily functioning in those with elevated scores, ensuring a balanced approach that combines ease of use with comprehensive symptom analysis. Integrating functional impairment assessment would thus significantly enhance HSCL's diagnostic accuracy. Surprisingly, few recent studies, both generally and specifically on university sam­ ples, have used structured diagnostic interviews, with the latest being over a decade old (Kou et al., 2012; Verger et al., 2010). The Dutch NEMESIS-3 study (ten Have et al., 2023), which monitors mental disorders in the Dutch general population, offers valuable insights. Using CIDI 3.0 (face-to-face interviews), the 12-month prevalence of any mental disorder was found to be 40% among young adults aged 18-24 years and 35% among those aged 25-35 years. Lifetime prevalence estimates were 50% and 59%, respectively. A sub-study from the 2020 HUNT study also provides relevant data (Knudsen et al., 2021). In this study, 2,154 participants from the general population were interviewed using CIDI 5.0. The 30-day prevalence of mental disorders among those aged 20-29 years was estimated to be 25.5% just before the COVID-19 pandemic. Although the prevalence estimates from NEMESIS-3 and HUNT are lower than those in our study of college and university students, they highlight the high prevalence of mental disorders among young adults. Some methodological considerations warrant attention. This study's reliance on the standardized and validated CIDI psychiatric survey represents a significant strength. However, the shift from traditional face-to-face interviews to a self-administered elec­ tronic format in CIDI 5.0 introduces challenges, such as the need for further validation against conventional methods. Past research (Knudsen et al., 2021) indicates no signif­ icant prevalence differences between face-to-face and telephone interviews, although Sivertsen, Skogen, Reneflot et al. 13 Clinical Psychology in Europe 2024, Vol. 6(4), Article e13275 https://doi.org/10.32872/cpe.13275 https://www.psychopen.eu/ recent comparisons between face-to-face and web-based self-reporting of psychological functioning revealed that respondents in face-to-face settings reported slightly fewer symptoms of depression (Cohen’s d = 0.25) (Kocjan et al., 2023). This raises questions about the accuracy of different administration modes, particularly as previous findings suggest that young, well-educated respondents might underreport mental health issues. Assessing the reliability of the self-administered CIDI, especially in comparison to face- to-face interviews, is crucial not only for validation but also for understanding how dif­ ferent modes might impact mental health assessments. However, discrepancies between HSCL and CIDI prevalence estimates, particularly among male respondents, require careful interpretation. These variations underscore the need for contextual adjustments and a deeper understanding of how assessment tools could produce divergent outcomes in mental health research. The proposed cut-offs, tailored for epidemiological research in a student population with unique demographic traits, might not generalize well to other groups due to varying baseline prevalences and cultural perceptions of mental health. Therefore, while HSCL shows promise as a screening tool in a student population, its broader applicability needs further validation. Moreover, the limited differences observed in mental health problems between CIDI respondents and non-respondents, along with consistent response rates across Norwegian regions, suggest a reasonable level geograph­ ic representativeness. However, our understanding of non-responders, limited to basic demographic details, restricts our ability to fully assess representativeness against the broader Norwegian student population. Recent findings from Denmark (Lyngsøe et al., 2023), showing minor participation variations across sociodemographic groups provide some reassurance regarding the generalizability of our results. The best approaches to determining cut-off values based on scales, like the various HSCL variants, depend on the intended purpose. If the goal is to identify individuals potentially in need of psychiatric treatment, prioritizing high sensitivity is crucial. How­ ever, when estimating the prevalence of depression or anxiety disorders within a popula­ tion, striking a balance between specificity and sensitivity relative to the true prevalence of the disorder might yield more accurate results. Recommending specific cut-off values for future studies lies beyond the scope of this study. Such norms can be effectively determined only after conducting several similar studies, ideally across diverse cultures, countries, and population segments. The benefits of using different cut-off points for men and women also warrant more comprehensive evaluation, compared against the challenges posed by adding such complexity. HSCL's Diagnostic Precision in Tertiary Students 14 Clinical Psychology in Europe 2024, Vol. 6(4), Article e13275 https://doi.org/10.32872/cpe.13275 https://www.psychopen.eu/ Funding: Norwegian Ministry of Education and Research. Acknowledgments: The authors have no additional (i.e., non-financial) support to report. Competing Interests: The authors have declared that no competing interests exist. Ethics Statement: The study was approved by the Regional Committee for Medical and Health Research Ethics in Western Norway (no. 2022/326437). All subjects gave electronic informed consent in accordance with the Declaration of Helsinki (2013). Preregistration: The study was not preregistered. The CIDI follow-up survey is registered at ClinicalTrials.gov (identifier: NCT05731102). Reporting Guidelines: We adhered to the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines in the design, conduct, and reporting of this study. Statement of the Conclusions: The study revealed high criterion validity for all HSCL versions in detecting major depressive episode and generalized anxiety disorder, suggesting their efficacy as screening tools rather than for precise prevalence estimation. Data Availability: Data are available upon reasonable request. All SHoT data set is administrated by the NIPH. Approval from a Norwegian regional committee for medical and health research ethics (https://helseforskning.etikkom.no) is a prerequisite. References American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Publishing. Auerbach, R. P., Alonso, J., Axinn, W. G., Cuijpers, P., Ebert, D. D., Green, J. G., Hwang, I., Kessler, R. C., Liu, H., Mortier, P., Nock, M. K., Pinder-Amaker, S., Sampson, N. A., Aguilar-Gaxiola, S., Al-Hamzawi, A., Andrade, L. H., Benjet, C., Caldas-de-Almeida, J. M., Demyttenaere, K., . . . Bruffaerts, R. (2016). Mental disorders among college students in the World Health Organization World Mental Health Surveys. Psychological Medicine, 46(14), 2955–2970. https://doi.org/10.1017/S0033291716001665 Chi, T., Cheng, L. Y., & Zhang, Z. J. (2023). Global prevalence and trend of anxiety among graduate students: A systematic review and meta-analysis. Brain and Behavior, 13(4), Article e2909. https://doi.org/10.1002/brb3.2909 Derogatis, L. R., Lipman, R. S., Rickels, K., Uhlenhuth, E. H., & Covi, L. (1974). The Hopkins Symptom Checklist (HSCL): A self-report symptom inventory. Behavioral Science, 19(1), 1–15. https://doi.org/10.1002/bs.3830190102 Sivertsen, Skogen, Reneflot et al. 15 Clinical Psychology in Europe 2024, Vol. 6(4), Article e13275 https://doi.org/10.32872/cpe.13275 https://helseforskning.etikkom.no https://doi.org/10.1017/S0033291716001665 https://doi.org/10.1002/brb3.2909 https://doi.org/10.1002/bs.3830190102 https://www.psychopen.eu/ Glaesmer, H., Braehler, E., Grande, G., Hinz, A., Petermann, F., & Romppel, M. (2014). The German Version of the Hopkins Symptoms Checklist-25 (HSCL-25) – Factorial structure, psychometric properties, and population-based norms. Comprehensive Psychiatry, 55(2), 396–403. https://doi.org/10.1016/j.comppsych.2013.08.020 Haro, J. M., Arbabzadeh-Bouchez, S., Brugha, T. S., de Girolamo, G., Guyer, M. E., Jin, R., Lepine, J. P., Mazzi, F., Reneses, B., Vilagut, G., Sampson, N. A., & Kessler, R. C. (2006). Concordance of the Composite International Diagnostic Interview Version 3.0 (CIDI 3.0) with standardized clinical assessments in the WHO World Mental Health surveys. International Journal of Methods in Psychiatric Research, 15(4), 167–180. https://doi.org/10.1002/mpr.196 Ibrahim, A. K., Kelly, S. J., Adams, C. E., & Glazebrook, C. (2013). A systematic review of studies of depression prevalence in university students. Journal of Psychiatric Research, 47(3), 391–400. https://doi.org/10.1016/j.jpsychires.2012.11.015 Jordanova, V., Wickramesinghe, C., Gerada, C., & Prince, M. (2004). Validation of two survey diagnostic interviews among primary care attendees: A comparison of CIS-R and CIDI with SCAN ICD-10 diagnostic categories. Psychological Medicine, 34(6), 1013–1024. https://doi.org/10.1017/S0033291703001727 Kessler, R. C., & Üstün, T. B. (2004). The World Mental Health (WMH) Survey Initiative version of the World Health Organization (WHO) Composite International Diagnostic Interview (CIDI). International Journal of Methods in Psychiatric Research, 13(2), 93–121. https://doi.org/10.1002/mpr.168 Kirkøen, B., Ørstavik, R., Reneflot, A., Skogen, J. C., Sivertsen, B., & Knudsen, A. K. S. (2024). The ability of the Hopkins Symptoms Checklist-5 to identify anxiety and depression in the general population [Manuscript in preparation]. Knapstad, M., Sivertsen, B., Knudsen, A. K., Smith, O. R. F., Aaro, L. E., Lonning, K. J., & Skogen, J. C. (2021). Trends in self-reported psychological distress among college and university students from 2010 to 2018. Psychological Medicine, 51(3), 470–478. https://doi.org/10.1017/S0033291719003350 Knudsen, A. K. S., Stene-Larsen, K., Gustavson, K., Hotopf, M., Kessler, R. C., Krokstad, S., Skogen, J. C., Øverland, S., & Reneflot, A. (2021). Prevalence of mental disorders, suicidal ideation and suicides in the general population before and during the COVID-19 pandemic in Norway: A population-based repeated cross-sectional analysis. The Lancet Regional Health – Europe, 4, Article 100071. https://doi.org/10.1016/j.lanepe.2021.100071 Kocjan, G. Z., Lavtar, D., & Sočan, G. (2023). The effects of survey mode on self-reported psychological functioning: Measurement invariance and latent mean comparison across face- to-face and web modes. Behavior Research Methods, 55(3), 1226–1243. https://doi.org/10.3758/s13428-022-01867-8 Kou, C., Meng, X. F., Xie, B., Chen, Y. F., Yu, Q., Shi, J. P., Yu, Y. Q., D’Arcy, C., & Huang, Y. Q. (2012). The prevalence and correlates of neurotic disorders among undergraduates at a mainland Chinese university. Social Psychiatry and Psychiatric Epidemiology, 47(12), 2011–2018. https://doi.org/10.1007/s00127-012-0500-2 HSCL's Diagnostic Precision in Tertiary Students 16 Clinical Psychology in Europe 2024, Vol. 6(4), Article e13275 https://doi.org/10.32872/cpe.13275 https://doi.org/10.1016/j.comppsych.2013.08.020 https://doi.org/10.1002/mpr.196 https://doi.org/10.1016/j.jpsychires.2012.11.015 https://doi.org/10.1017/S0033291703001727 https://doi.org/10.1002/mpr.168 https://doi.org/10.1017/S0033291719003350 https://doi.org/10.1016/j.lanepe.2021.100071 https://doi.org/10.3758/s13428-022-01867-8 https://doi.org/10.1007/s00127-012-0500-2 https://www.psychopen.eu/ Lyngsøe, S., Lophaven, S., Jepsen, R., Holmager, T., Janssens, A., & Lynge, E. (2023). Non- participation in a health examination survey in a rural-provincial area of Denmark – Results from the Lolland-Falster Health Study (LOFUS). Scandinavian Journal of Public Health. Advance online publication. https://doi.org/10.1177/14034948231206879 McCloud, T., Kamenov, S., Callender, C., Lewis, G., & Lewis, G. (2023). The association between higher education attendance and common mental health problems among young people in England: Evidence from two population-based cohorts. The Lancet Public Health, 8(10), e811– e819. https://doi.org/10.1016/S2468-2667(23)00188-3 McKnight, P. E., Monfort, S. S., Kashdan, T. B., Blalock, D. V., & Calton, J. M. (2016). Anxiety symptoms and functional impairment: A systematic review of the correlation between the two measures. Clinical Psychology Review, 45, 115–130. https://doi.org/10.1016/j.cpr.2015.10.005 Nordgaard, J., Sass, L. A., & Parnas, J. (2013). The psychiatric interview: Validity, structure, and subjectivity. European Archives of Psychiatry and Clinical Neuroscience, 263(4), 353–364. https://doi.org/10.1007/s00406-012-0366-z Rettew, D. C., Lynch, A. D., Achenbach, T. M., Dumenci, L., & Ivanova, M. Y. (2009). Meta-analyses of agreement between diagnoses made from clinical evaluations and standardized diagnostic interviews. International Journal of Methods in Psychiatric Research, 18(3), 169–184. https://doi.org/10.1002/mpr.289 Rodríguez-Barragán, M., Fernández-San-Martín, M. I., Clavería, A., Le Reste, J. Y., Nabbe, P., Motrico, E., Gómez-Gómez, I., & Peguero-Rodríguez, E. (2023). Measuring depression in Primary Health Care in Spain: Psychometric properties and diagnostic accuracy of HSCL-5 and HSCL-10. Frontiers in Medicine, 9, Article 1014340. https://doi.org/10.3389/fmed.2022.1014340 Rodríguez-Barragán, M., Fernández-San-Martín, M. I., Clavería-Fontán, A., Aldecoa-Landesa, S., Casajuana-Closas, M., Llobera, J., Oliván-Blázquez, B., & Peguero-Rodríguez, E. (2021). Validation and Psychometric Properties of the Spanish Version of the Hopkins Symptom Checklist-25 Scale for Depression Detection in Primary Care. International Journal of Environmental Research and Public Health, 18(15), Article 7843. https://doi.org/10.3390/ijerph18157843 Sandanger, I., Moum, T., Ingebrigtsen, G., Dalgard, O. S., Sørensen, T., & Bruusgaard, D. (1998). Concordance between symptom screening and diagnostic procedure: The Hopkins Symptom Checklist-25 and the Composite International Diagnostic Interview I. Social Psychiatry and Psychiatric Epidemiology, 33(7), 345–354. https://doi.org/10.1007/s001270050064 Sheldon, E., Simmonds-Buckley, M., Bone, C., Mascarenhas, T., Chan, N., Wincott, M., Gleeson, H., Sow, K., Hind, D., & Barkham, M. (2021). Prevalence and risk factors for mental health problems in university undergraduate students: A systematic review with meta-analysis. Journal of Affective Disorders, 287, 282–292. https://doi.org/10.1016/j.jad.2021.03.054 Sivertsen, B., Knudsen, A. K. S., Kirkøen, B., Skogen, J. C., Lagerstrøm, B. O., Lønning, K.-J., Kessler, R. C., & Reneflot, A. (2023). Prevalence of mental disorders among Norwegian college and university students: A population-based cross-sectional analysis. The Lancet Regional Health – Europe, 34, Article 100732. https://doi.org/10.1016/j.lanepe.2023.100732 Sivertsen, Skogen, Reneflot et al. 17 Clinical Psychology in Europe 2024, Vol. 6(4), Article e13275 https://doi.org/10.32872/cpe.13275 https://doi.org/10.1177/14034948231206879 https://doi.org/10.1016/S2468-2667(23)00188-3 https://doi.org/10.1016/j.cpr.2015.10.005 https://doi.org/10.1007/s00406-012-0366-z https://doi.org/10.1002/mpr.289 https://doi.org/10.3389/fmed.2022.1014340 https://doi.org/10.3390/ijerph18157843 https://doi.org/10.1007/s001270050064 https://doi.org/10.1016/j.jad.2021.03.054 https://doi.org/10.1016/j.lanepe.2023.100732 https://www.psychopen.eu/ Sivertsen, B., Råkil, H., Munkvik, E., & Lonning, K. J. (2019). Cohort profile: The SHoT-study, a national health and well-being survey of Norwegian university students. BMJ Open, 9(1), Article e025200. https://doi.org/10.1136/bmjopen-2018-025200 Skogen, J. C., Overland, S., Smith, O. R. F., & Aaro, L. E. (2017). The factor structure of the Hopkins Symptoms Checklist (HSCL-25) in a student population: A cautionary tale. Scandinavian Journal of Public Health, 45(4), 357–365. https://doi.org/10.1177/1403494817700287 ten Have, M., Tuithof, M., van Dorsselaer, S., Schouten, F., Luik, A. I., & de Graaf, R. (2023). Prevalence and trends of common mental disorders from 2007-2009 to 2019-2022: Results from the Netherlands Mental Health Survey and Incidence Studies (NEMESIS), including comparison of prevalence rates before vs. during the COVID-19 pandemic. World Psychiatry, 22(2), 275–285. https://doi.org/10.1002/wps.21087 Veijola, J., Jokelainen, J., Läksy, K., Kantojärvi, L., Kokkonen, P., Järvelin, M. R., & Joukamaa, M. (2003). The Hopkins Symptom Checklist-25 in screening DSM-III-R axis-I disorders. Nordic Journal of Psychiatry, 57(2), 119–123. https://doi.org/10.1080/08039480310000941 Verger, P., Guagliardo, V., Gilbert, F., Rouillon, F., & Kovess-Masfety, V. (2010). Psychiatric disorders in students in six French universities: 12-month prevalence, comorbidity, impairment and help- seeking. Social Psychiatry and Psychiatric Epidemiology, 45(2), 189–199. https://doi.org/10.1007/s00127-009-0055-z Clinical Psychology in Europe (CPE) is the official journal of the European Association of Clinical Psychology and Psychological Treatment (EACLIPT). PsychOpen GOLD is a publishing service by Leibniz Institute for Psychology (ZPID), Germany. HSCL's Diagnostic Precision in Tertiary Students 18 Clinical Psychology in Europe 2024, Vol. 6(4), Article e13275 https://doi.org/10.32872/cpe.13275 https://doi.org/10.1136/bmjopen-2018-025200 https://doi.org/10.1177/1403494817700287 https://doi.org/10.1002/wps.21087 https://doi.org/10.1080/08039480310000941 https://doi.org/10.1007/s00127-009-0055-z https://www.psychopen.eu/ HSCL's Diagnostic Precision in Tertiary Students (Introduction) Method Setting and Participants Instruments Statistical Analyses Results Discussion (Additional Information) Funding Acknowledgments Competing Interests Ethics Statement Preregistration Reporting Guidelines Statement of the Conclusions Data Availability References