Advances in Social Science and Culture ISSN 2640-9682 (Print) ISSN 2640-9674 (Online) Vol. 6, No. 3, 2024 www.scholink.org/ojs/index.php/assc 134 Original Paper Reliability and Validity of the Chinese Version of the Threat Appraisal Questionnaire in Assessing Children and Adolescents Jiarui PAN1* 1 School of Psychology，Guizhou Normal University y, Guiyang Received: April 27, 2024 Accepted: June 11, 2024 Online Published: June 20, 2024 doi:10.22158/assc.v6n3p134 URL: http://dx.doi.org/10.22158/assc.v6n3p134 Abstract This study introduces the Threat Appraisal Questionnaire for Children (TAQ-C) and examines its reliability and validity among Chinese children and adolescents. The TAQ-C was administered to 903 children and adolescents, with PSWQ-C, IUSC-12 and MASC used as criterion tools. After two weeks, 38 adolescents completed the TAQ-C retest. The revised Chinese version of the Threat Appraisal Questionnaire presents a three-factor structure, including perceived probability, perceived severity, and coping cost, comprising 36 items. The model fit indices were satisfactory (χ2/df=3.521, RMSEA=0.053, CFI=0.914, TLI=0.906, SRMR=0.048). Multigroup confirmatory factor analysis further demonstrated that the revised TAQ-C meets measurement invariance across genders. The TAQ-C scores showed significant correlations with the scores of the criterion questionnaires, indicating good criterion validity. The revised questionnaire demonstrated good reliability, with an overall alpha coefficient of 0.93 for the full scale, a retest reliability of 0.76, and alpha coefficients for the subscales ranging from 0.81 to 0.87. The retest reliability of the subscales ranged from 0.60 to 0.79. The revised Chinese version of the Threat Appraisal Questionnaire shows a three-factor structure and demonstrates overall good reliability and validity among Chinese children and adolescents. Keywords Threat Appraisal, Anxiety, Reliability, Validity 1. Introduction According to the "Report on the Development of Chinese Youth," over 30 million children and adolescents under the age of 17 in China experience issues such as depression and anxiety. The government places a high priority on the mental health of adolescents, incorporating it into the "Healthy China" strategy and issuing a series of policy documents. In July 2021, the Ministry of Education released a notice on strengthening student mental health management, explicitly proposing www.scholink.org/ojs/index.php/assc Advances in Social Science and Culture Vol. 6 No. 3, 2024 Published by SCHOLINK INC. 135 "to improve mental health assessment work" (Liu, 2023). Anxiety is one of the most common emotional problems among adolescents (Sun, Ju, Jiang et al., 2018), typically arising in threatening environments (Eysenck Derakshan, Santos & Calvo, 2007). It refers to the physiological, emotional, and behavioral tension, unease, and worry individuals experience in response to potentially threatening situations (William WK, Zung, 1971). Anxiety affects adolescents' learning and life, leading to interpersonal communication barriers and negatively impacting their quality of life (Knapp, Blumenthal, Mischel et al., 2016; Xing, Wang, Wei et al., 2016). When facing potentially threatening situations, anxious individuals are more likely to perceive the threat significance of uncertain events, and anxiety is associated with increased attention to threat cues (Mathews & MacLeod, 1994). This involves a cognitive appraisal process of the threat stimuli, known as threat appraisal (Mathews & Mackintosh, 1998; Li, 2022; Aaron, 1985). Threat appraisal broadly refers to an individual's cognitive judgment of a threat information or event, encompassing perceived probability, perceived severity, and coping cost (Francis, Hawes, Abbott et al., 2017; Fan, 2020). Perceived probability refers to the individual's assessment of the likelihood of a threat event occurring to themselves. Perceived severity refers to the individual's judgment of the seriousness of the consequences of the threat event or the extent of harm to themselves, i.e., the extent of the event's severity. Coping cost refers to the difficulty of taking measures when facing a threat (Clark & Beck, 2011; Dugas, Letarte, Rhéaume et al., 1995; Wells, 1995; Janis, 1967). In ambiguous situations, individuals with high anxiety traits are more likely to perceive the situation as threatening compared to those with low anxiety traits (Mogg & Bradley, 1998; Eysenck, Mogg, May et al., 1991). This cognitive process of threat appraisal influences the development and maintenance of anxiety (Craske, Rapee, Jackel et al., 1989; Mathews, Richards & Eysenck, 1989). Therefore, the ability to measure different dimensions of threat appraisal through self-report can help in studying the cognitive processes underlying children's anxiety. There are many studies and various scales developed domestically and internationally for measuring anxiety traits in children and adolescents. However, there is still a lack of self-report tools specifically measuring threat appraisal, a crucial cognitive process closely related to anxiety. To measure the cognitive characteristic of threat appraisal, Francis et al. developed the Threat Appraisal Questionnaire for Children (TAQ-C) (Aaron, 1985; Stapinski, Abbott, & Rapee, 2010). The TAQ-C comprises 36 items and measures individuals' cognitive tendencies in assessing threats from three dimensions: (1) probability of the event occurring, (2) the severity or painfulness of the consequences (cost), and (3) the difficulty of coping (coping). The TAQ-C has demonstrated good reliability and validity among children and adolescents in foreign countries. Currently, there is no similar scale for threat appraisal in China, and it is unclear whether the TAQ-C has cross-cultural structural consistency. Therefore, this study aims to introduce the TAQ-C into China and examine its reliability and validity among Chinese children and adolescents, providing a reliable, simple, and practical tool for conducting threat appraisal research domestically. www.scholink.org/ojs/index.php/assc Advances in Social Science and Culture Vol. 6 No. 3, 2024 Published by SCHOLINK INC. 136 2. Subjects and Methods 2.1 Subjects Sample One: A convenient sampling method was used to survey students from two primary and secondary schools in Guizhou Province. Questionnaires were distributed and collected uniformly during extracurricular time, with a total of 950 paper questionnaires distributed. After excluding questionnaires with patterned responses or excessive missing answers, 903 valid questionnaires were retained, resulting in an effective rate of 95.1%. Among the respondents, 483 were male (53.5%) and 420 were female (46.5%), with an average age of 13.22±1.49 years. The sample included 449 primary school students and 454 secondary school students. Sample Two: A total of 45 retest questionnaires were distributed, and 38 were collected two weeks later, with a recovery rate of 84.4%. Among the respondents, 22 were male (57.9%) and 16 were female (42.1%), aged between 13 and 14 years. 2.2 Instruments 2.2.1 Threat Appraisal Questionnaire The questionnaire consists of 12 situational items, divided into three subscales: perceived probability, perceived severity, and coping cost. Each subscale item is the same, and respondents need to rate each situational item on an 8-point scale from three dimensions: (1) How likely is it that this event will happen to you in the future? (perceived probability); (2) If this event happens to you, how bad will it be? (perceived severity); (3) If this event happens to you, how difficult will it be to deal with? (coping cost). Scores for each subscale range from 12 to 96, and the total score for the questionnaire ranges from 36 to 288, with higher scores indicating a higher perception of the likelihood, negative consequences, and difficulty in coping with future threatening events. With the original author's permission, the questionnaire was revised and adapted into Chinese. After obtaining authorization, the questionnaire was initially translated by a psychology student. The translation was then discussed and revised within the group to ensure accurate expression of the original content and to conform to Chinese language habits. After repeated discussions, the final version was formed and reviewed by a psychology professor. The items and scoring method are consistent with the original questionnaire. 2.2.2 Criterion Questionnaires The Penn State Worry Questionnaire for Children (PSWQ-C) (Chorpita, Tracey, Brown et al., 1997), developed by Chorpita et al., consists of 14 items and uses a 4-point Likert scale ranging from 1 ("not at all") to 4 ("always"), with higher scores indicating more pronounced anxiety traits. The Intolerance of Uncertainty Scale for Children (IUSC-12) (Wu, Wang, & Qi, 2016), the Chinese version revised by Zhang Yajuan et al. (2017), includes 12 items with two dimensions: prospective anxiety and inhibitory anxiety. The prospective anxiety dimension includes 7 items, such as "Unpredictable things upset me"; the inhibitory anxiety dimension includes 5 items, such as "When it's time to act, uncertainty stops me". The scale uses a 5-point scoring method, from 1 ("not at all www.scholink.org/ojs/index.php/assc Advances in Social Science and Culture Vol. 6 No. 3, 2024 Published by SCHOLINK INC. 137 characteristic") to 5 ("entirely characteristic"), with the total score being the sum of the two dimensions. Higher total scores indicate higher levels of intolerance of uncertainty. The Multidimensional Anxiety Scale for Children (MASC) [24], consists of 39 items rated on a 4-point Likert scale from 0 ("not at all true") to 3 ("very true"), assessing anxiety in four aspects over the past week: emotional, cognitive, behavioral, and physical symptoms. It includes four subscales: (1) physical symptoms with 12 items; (2) social anxiety with 9 items; (3) separation anxiety with 9 items; and (4) harm avoidance with 9 items. The total score ranges from 0 to 117, with higher scores indicating higher levels of anxiety. The Chinese version of this scale has good reliability and validity and can be used to assess anxiety levels in Chinese children and adolescents. 2.3 Data Processing and Analysis Methods Statistical analyses were conducted using SPSS 26.0 and AMOS 24.0 software. Item analysis included independent sample t-tests, critical ratio (CR) analysis, and correlations between item scores and total scores. The structural validity of the questionnaire was evaluated using exploratory and confirmatory factor analysis. Reliability was assessed using Cronbach's alpha coefficient and test-retest reliability. Confirmatory factor analysis was conducted using AMOS 24.0, while other analyses were performed using SPSS 26.0. 3. Results 3.1 Item Analysis The Pearson correlation method was used to analyze the correlation between each item score and the total score of the TAQ-C. The results showed that all item scores were significantly correlated with the total score (r=0.34~0.66, p<0.01). Each item score was also significantly correlated with the total score of its respective dimension (perceived probability, r=0.34~0.50, p<0.01; perceived severity, r=0.48~0.66, p<0.01; coping cost, r=0.43~0.65, p<0.01). The sample was divided into high and low scoring groups based on the top and bottom 27% of the total score distribution, resulting in a high score group (246 individuals) and a low score group (244 individuals). An independent sample t-test was then used to compare the differences in item scores between the high and low scoring groups. The t-values ranged from 10.39 to 25.37, all with p<0.01, indicating statistically significant differences. 3.2 Structural Validity Confirmatory factor analysis (CFA) was conducted using Amos, and the fitting results are shown in Table 1. First, CFA was performed based on the original model. There was a high residual correlation between dimension two (perceived severity) and dimension three (coping cost), with each item in these dimensions corresponding pairwise (e.g., "perceived severity" item 1 corresponds to "coping cost" item 1). After adding residual correlations between items, CFA was conducted again, as shown in Model 1 in Table 1. Based on the modification indices, residual correlations were also found between item 12 of dimension one (perceived probability) and items 12 of dimensions two and three. After modification, CFA was conducted again, and the results are shown in Model 2 in Table 1. The fit indices of Model 2 www.scholink.org/ojs/index.php/assc Advances in Social Science and Culture Vol. 6 No. 3, 2024 Published by SCHOLINK INC. 138 were better than those of Model 1, so Model 2 was adopted as the final model. The factor loadings of each item in Model 2 ranged from 0.38 to 0.67, as detailed in Table 2. Table 1. Comparison of Fit Indices for Confirmatory Factor Analysis Models Model χ2 /df RMSEA CFI TLI SRMR Model 1 3.81 0.056 0.904 0.896 0.051 Model 2 3.52 0.053 0.914 0.906 0.048 Table 2. Factor Loadings of Each Item in the Chinese Version of the TAQ-C Item Dimension 1 Dimension 2 Dimension 3 Possible1 0.42 Possible2 0.50 Possible3 0.40 Possible4 0.38 Possible5 0.47 Possible6 0.50 Possible7 0.59 Possible8 0.58 Possible9 0.63 Possible10 0.48 Possible11 0.61 Possible12 0.54 Terrible1 0.45 Terrible2 0.60 Terrible3 0.61 Terrible4 0.60 Terrible5 0.61 Terrible6 0.55 Terrible7 0.56 Terrible8 0.65 Terrible9 0.64 Terrible10 0.61 Terrible11 0.67 Terrible12 0.51 Copy1 0.41 Copy2 0.57 www.scholink.org/ojs/index.php/assc Advances in Social Science and Culture Vol. 6 No. 3, 2024 Published by SCHOLINK INC. 139 Copy3 0.63 Copy4 0.60 Copy5 0.59 Copy6 0.58 Copy7 0.59 Copy8 0.68 Copy9 0.64 Copy10 0.64 Copy11 0.64 Copy12 0.52 Figure 1. Fit of the Three-Factor Model of the TAQ-C in Confirmatory Factor Analysis 3.3 Criterion-Related Validity The Penn State Worry Questionnaire for Children (PSWQ-C), the Intolerance of Uncertainty Scale (short version) (IUSC-12), and the Multidimensional Anxiety Scale for Children (MASC) were www.scholink.org/ojs/index.php/assc Advances in Social Science and Culture Vol. 6 No. 3, 2024 Published by SCHOLINK INC. 140 selected as criterion-related validity indicators for the TAQ-C. Pearson correlation analysis showed that the total and factor scores of the TAQ-C were significantly positively correlated with the total scores of the PSWQ-C, IUSC-12, and MASC, with correlation coefficients ranging from 0.33 to 0.43 (p<0.01), as detailed in Table 3. Table 3. Criterion-Related Validity Analysis of the TAQ-C PSWQ-C IUSC-12 MASC TAQ-C 0.41 0.43 0.43 F1 0.33 0.35 0.36 F2 0.38 0.41 0.40 F3 0.37 0.38 0.38 Note: p<0.01. 3.4 Reliability Analysis 3.4.1 Internal Consistency Reliability The reliability of the TAQ-C scale was analyzed. The Cronbach’s alpha coefficient for the total scale was 0.93, and for the three dimensions (perceived probability, perceived severity, and coping cost), the Cronbach’s alpha coefficients were 0.81, 0.86, and 0.87, respectively. 3.4.2 Test-Retest Reliability Using data from Sample Two, a two-week test-retest reliability analysis was conducted. The Intraclass Correlation Coefficient (ICC) for the total scale was 0.76, and for the three factors, the ICC values were 0.60 (perceived probability), 0.78 (perceived severity), and 0.79 (coping cost). 3.5 Measurement Invariance Amos was used to test the measurement invariance of the three-factor model of the TAQ-C across gender groups. The results showed that the indices of the four models (configural invariance, metric invariance, scalar invariance, and residual invariance) all met the basic psychometric requirements, and the ΔGFI and ΔRMSEA values for each step of the measurement invariance test were less than 0.01. These results indicate that the revised TAQ-C has measurement invariance across genders. Table 3. Measurement Invariance Test Across Genders Model CMIN df SRMR GFI RMSEA ΔSRMR ΔGFI ΔRMSEA Configural Invariance 3001.71 1158 0.054 0.846 0.042 Metric Invariance 3036.32 1191 0.056 0.845 0.041 0.002 0.001 0.001 Scalar 3059.05 1197 0.057 0.843 0.042 0.001 0.002 0.001 www.scholink.org/ojs/index.php/assc Advances in Social Science and Culture Vol. 6 No. 3, 2024 Published by SCHOLINK INC. 141 Invariance Residual Invariance 3272.46 1245 0.059 0.835 0.043 0.002 0.008 0.001 4. Conclusion This study revised and tested the reliability, validity, and measurement invariance of the TAQ-C in a sample of Chinese children and adolescents, aiming to provide a simplified and reliable measurement tool for threat appraisal research in the Chinese cultural context. Item analysis of the TAQ-C showed that all items significantly differentiated between high and low scoring groups, indicating good item discrimination. Additionally, all items' critical ratios (CR) were significant, and item-total score correlation coefficients ranged from 0.34 to 0.66, indicating good item quality and suitability for further analysis. Confirmatory factor analysis (CFA) results indicated that the TAQ-C is a three-factor model, including perceived probability, perceived severity, and coping cost. The model's structure aligned with the original model and supported the cognitive model of anxiety with three cognitive appraisal dimensions. However, the original model's fit was not ideal for the Chinese sample, leading to the addition of residual correlations between specific items. CFA supported the revised three-factor model, with all fit indices meeting relevant requirements, demonstrating the structural validity of the Chinese version of the TAQ-C. Reliability analysis showed that the Cronbach’s alpha coefficients for the total scale and each dimension ranged from 0.81 to 0.93, consistent with the results of the English version, indicating good internal consistency. The two-week test-retest reliability ranged from 0.60 to 0.79, indicating high stability over time.Criterion-related validity analysis showed that TAQ-C scores were positively correlated with the scores of the Chinese versions of the PSWQ-C, IUSC-12, and MASC, indicating good criterion-related validity. This result suggests that the three cognitive appraisal processes of threat appraisal are positively correlated with individual anxiety levels. Individuals with higher anxiety levels are more likely to perceive the threat significance of uncertain events and have increased attention to threat cues. This study confirms the close relationship between cognitive characteristics of threat appraisal and anxiety levels. Therefore, the TAQ-C can be used to further screen individuals with potential anxiety traits among children and adolescents and to compare changes in threat appraisal before and after anxiety treatment.Furthermore, this study tested the gender invariance of the TAQ-C in Chinese children and adolescents. The three-factor structure of the TAQ-C fit well for both male and female samples, meeting basic psychometric requirements. This indicates that the revised TAQ-C functions equivalently across genders in measuring threat appraisal.The study sample only included children and adolescents from Guizhou Province, which is relatively homogeneous. Future research should include samples from different regions to further verify the reliability and validity of the TAQ-C in Chinese children and adolescents. In this study, the reliability and test-retest reliability of the perceived probability dimension were slightly lower than those of the perceived severity, coping cost www.scholink.org/ojs/index.php/assc Advances in Social Science and Culture Vol. 6 No. 3, 2024 Published by SCHOLINK INC. 142 dimensions, and the total threat appraisal scale. This may be due to the inapplicability of certain traumatic scenarios in the questionnaire to the sample, such as "finding out you need to have a tooth pulled during a routine dental check-up" or "your pet running away because you forgot to close the door". Routine dental visits and pet ownership may not be common in the tested population, affecting the reliability of responses to these items. This could be due to differences in regional living habits or economic development levels. However, perceived probability is an essential part of the cognitive process of threat appraisal, and for model symmetry, all items were retained without deletion. Overall, this study confirms that the Chinese version of the TAQ-C has good reliability and validity among Chinese children and adolescents. It can be directly used for gender comparisons and provides a scientific, reliable, and convenient tool for assessing threat appraisal in Chinese children and adolescents. References Aaron, T. B. (1985). Anxiety disorders and phobias: a cognitive perspective. New York: Basic Books. Chorpita, B. F., Tracey, S. A., Brown, T. A. et al. (1997). Assessment of worry in children and adolescents: An adaptation of the Penn State Worry Questionnaire. Behaviour Research and Therapy, 35(6), 569-581. https://doi.org/10.1016/S0005-7967(96)00116-7 Clark, D. A., & Beck, A. T. (2011). Cognitive therapy of anxiety disorders: Science and practice. New York: Guilford Press. Craske, M. G., Rapee, R. M., Jackel, L. et al. (1989). Qualitative dimensions of worry in DSM-III-R generalized anxiety disorder subjects and nonanxious controls. Behavior Research and Therapy, 27(4), 397-402. https://doi.org/10.1016/0005-7967(89)90010-7 Dugas, M. J., Letarte, H., Rhéaume, J. et al. (1995). Worry and problem-solving: Evidence of a specific relationship. Cognitive Therapy and Research, 19(1), 109-120. https://doi.org/10.1007/BF02229679 Eysenck, M. W., Derakshan, N., Santos, R., & Calvo, M. G. (2007) Anxiety and cognitive performance: Attentional control theory. Emotion, 7(2), 336-353. https://doi.org/10.1037/1528-3542.7.2.336 Eysenck, M. W., Mogg, K., May, J. et al. (1991). Bias in interpretation of ambiguous sentences related to threat in anxiety. Journal of Abnormal Psychology, 100(2), 144-150. https://doi.org/10.1037/0021-843X.100.2.144 Fan, C. (2020). Review of Protection Motivation Theory Research. Modern Business and Industry, 41(16), 79-81. Francis, R., Hawes, D. J., Abbott, M. et al. (2017). Development and Preliminary Validation of the Threat Appraisal Questionnaire for Children (TAQ-C). J Psychopathol Behav Assess, 39, 322-341. https://doi.org/10.1007/s10862-016-9584-4 Janis, I. L. (1967). Effects of Fear Arousal on Attitude Change: Recent Developments in Theory and Experimental Research1. Advances in Experimental Social Psychology, 3, 166-224. www.scholink.org/ojs/index.php/assc Advances in Social Science and Culture Vol. 6 No. 3, 2024 Published by SCHOLINK INC. 143 https://doi.org/10.1016/S0065-2601(08)60344-5 Knapp, A. A., Blumenthal, H., Mischel, E. R. et al. (2016). Anxiety Sensitivity and Its Factors in Relation to Generalized Anxiety Disorder among Adolescents. J Abnorm Child Psychol, 44(2), 233-244. https://doi.org/10.1007/s10802-015-9991-0 Li, W. J. (2022). The Impact of Dynamic Threatening Emotional Body Language on Preschoolers' Attention Bias: The Mediating Role of Threat Appraisal. Zhejiang Normal University. Liu, C. (2023). Collaboration and Disorder: The “Over-Disease” of Adolescent Depression. Contemporary Youth Research, (02), 74-84. Mathews, A., & Mackintosh, B. (1998). A cognitive model of selective processing in anxiety. Cognitive Therapy and Research, 22(6), 539-560. https://doi.org/10.1023/A:1018738019346 Mathews, A., & MacLeod, C. (1994). Cognitive approaches to emotion and emotional disorders. Annu Rev Psychol, 45, 25-50. https://doi.org/10.1146/annurev.ps.45.020194.000325 Mathews, A., Richards, A., & Eysenck, M. (1989). Interpretation of homophones related to threat in anxiety states. Journal of Abnormal Psychology, 98(1), 31-34. https://doi.org/10.1037/0021-843X.98.1.31 Mogg, K., & Bradley, B. P. (1998). A cognitive-motivational analysis of anxiety. Behaviour Research and Therapy, 36(9), 809-848. https://doi.org/10.1016/S0005-7967(98)00063-1 Stapinski, L., Abbott, M. J., & Rapee, R. M. (2010). Fear and perceived uncontrollability of emotion: Evaluating the unique contribution of emotion appraisal variables to prediction of worry and generalized anxiety disorder. Behaviour Research and Therapy, 48(11), 1097-1104. https://doi.org/10.1016/j.brat.2010.07.012 Sun, L. P., Ju, J. W., Jiang, L. Q. et al. (2018). The Relationship Between Parental Psychological Control and Adolescent Anxiety: A Cross-Lagged Study. Psychological Development and Education, 34(06). 758-768. Tan, F. R. (2011). Measurement Invariance Study of the Multidimensional Anxiety Scale for Children in Chinese and Italian Children Samples. Central South University. Wells, A. (1995). Meta-cognition and worry: A cognitive model of generalized anxiety disorder. Behavioural and Cognitive Psychotherapy, 23(3), 301-320. https://doi.org/10.1017/S1352465800015897 William, W. K., & Zung. A. (1971). Rating Instrument for Anxiety Disorders. Psychosomatics, 12(6), 371-379. https://doi.org/10.1016/S0033-3182(71)71479-0 Wu, L. J., Wang, J. N., & Qi, X. D. (2016). Validity and Reliability of the Short Version of the Intolerance of Uncertainty Scale in Middle School Students. Chinese Journal of Mental Health, 30(09), 700-705. Xing, Y. L., Wang, J. P., Wei, W. et al. (2016). The Impact of Social Support on Adolescent Anxiety: The Mediating Role of Emotion Regulation Strategies. Chinese Journal of Clinical Psychology, 24(06), 1079-1082. 1.Introduction 2.Subjects and Methods 2.1 Subjects 2.2 Instruments 2.2.1 Threat Appraisal Questionnaire 2.2.2 Criterion Questionnaires 2.3 Data Processing and Analysis Methods 3.Results 3.1 Item Analysis 3.2 Structural Validity 3.3 Criterion-Related Validity 3.4 Reliability Analysis 3.4.1 Internal Consistency Reliability 3.4.2 Test-Retest Reliability 3.5 Measurement Invariance 4.Conclusion References Liu, C. (2023). Collaboration and Disorder: The “O