Terzi, R. (2020). An adaptation of artificial intelligence anxiety scale into Turkish: Reliability and validity study. International Online Journal of Education and Teaching (IOJET), 7(4). 1501-1515. http://iojet.org/index.php/IOJET/article/view/1031 Received: 10.07.2020 Received in revised form: 15.08.2020 Accepted: 09.09.2020 AN ADAPTATION OF ARTIFICIAL INTELLIGENCE ANXIETY SCALE INTO TURKISH: RELIABILITY AND VALIDITY STUDY Research Article Ragip Terzi Harran University terziragip@harran.edu.tr Ragip Terzi is an assistant professor at Harran University, Turkey. His research interests focus on applied statistics and psychometrics. His publications appeared in Applied Psychological Measurement, Journal of Educational Measurement, and Journal of Measurement and Evaluation in Education and Psychology. Copyright by Informascope. Material published and so copyrighted may not be published elsewhere without the written permission of IOJET. mailto:terziragip@harran.edu.tr http://orcid.org/0000-0003-3976-5054 International Online Journal of Education and Teaching (IOJET) 2020, 7(4), 1501-1515. 1501 AN ADAPTATION OF ARTIFICIAL INTELLIGENCE ANXIETY SCALE INTO TURKISH: RELIABILITY AND VALIDITY STUDY1 Ragip Terzi terziragip@harran.edu.tr Abstract The widespread use of artificial intelligence (AI) has been growing in various fields. AI is defined as human-like automation in place of human beings that can operate many functions based on some level of intelligence. In education, AI offers powerful pedagogical tools that can help enhance instructional quality. Given the inevitable advancements of AI in education, this study aims to investigate teachers’ AI anxiety levels based on various demographic factors. For this purpose, the AI Anxiety Scale is adapted into Turkish, which provides a good fit of the model to the data for the construct validity. Moreover, the reliability coefficients of the scale show strong evidence of consistency in teachers’ responses to the items. For sociotechnical blindness dimension, male and female teachers do not show any significant differences. However, for learning, job replacement, AI configuration dimensions and the total scale, female teachers are more anxious towards AI than male teachers. Moreover, there is no difference observed based on degree levels teachers hold. Additionally, anxiety levels of teachers are not related to teachers’ age and years of experience in teaching. Keywords: anxiety, artificial intelligence, scale adaptation, validity and reliability, teachers 1. Introduction Nowadays, the widespread use of artificial intelligence (AI) has been growing in various fields such as healthcare, engineering, finance, marketing, banking, agriculture, law, and education. AI is defined as human-like automation in place of human beings that can operate many functions based on some level of intelligence such as responding to questions, coping with emerging issues, figuring out problems, and likewise (Coppin, 2004). For the advantage of operating AI in different sectors, McKinsey Global Institute (MGI) reported that 3 to 14% of workers (i.e., 75 to 375 million people) may need to enhance their abilities and/or switch their professions until 2030 (Manyika et al., 2017). Wang and Siau (2019) also pointed out the expeditious enhancement in AI technology that can replace many professions. This situation forces people to adapt working with AI technologies and products, which will eventually require them to get properly prepared to fulfill relevant employment needs in the future. It is still inevitable that computerization and automation may take over human work due to being largely dependent on AI technologies (Nauman, 2017). Moreover, people whose contributions to technology have been well-recognized (e.g., Bill Gates, Elon Musk, and Stephen Hawking) also noted that AI could have negative impacts on people and society in unfortunate ways if it gets out of control (Future of Life Institute [FLI], 1 This study was supported in part by the European Union Education and Youth Programs Center Erasmus+ Project, entitled “Artificial Intelligence Education for Children” and numbered 2019-1-TR01-KA201-077041 of Harran University. The European Commission and the National Agency of Turkey cannot be held responsible for the opinions expressed in this study. mailto:terziragip@harran.edu.tr Terzi 1502 2015). These concerns have led researchers to study on the perception and adoption of individuals about computer anxiety (e.g. Chuo, Tsai, Lan, & Tsai, 2011; Esterhuyse, Scholtz, & Venter, 2016; Korobili, Togia, & Malliari, 2010; Marcoulides, 1989), mobile computer anxiety (Wang, 2007), Internet anxiety (Chou, 2003), and robot anxiety (Nomura, Suzuki, Kanda, & Kato, 2006; Wu et al., 2014). Despite varying anxiety levels among people, conventional measurement tools have not been commonly developed to measure anxiety levels of individuals for AI technologies. Anxiety towards AI technologies can occur due to imprecise attitudes towards technological enhancement, bewilderment about autonomy, and sociotechnical blindness (Johnson & Verdicchio, 2017; Wang & Wang, 2019). In other words, “AI anxiety (AIA)” can be expressed as the panic and nervousness due to unknown directions of AI development (Johnson & Verdicchio, 2017). Because of a need for properly designed tools to measure AIA levels of individuals, Wang and Wang (2019) recently developed an AIA scale with four factors (i.e., sub-dimensions), namely, learning, job replacement, sociotechnical blindness, and AI configuration. Learning dimension similar to computer-anxiety construct is used to measure how much anxious people are with the learning the applications of AI techniques and products in their career. In the era of technology, learning AI-related technologies is crucial to stay in the profession because employees can be constantly required to fulfill relevant skills. Another dimension is job replacement that is used to measure anxiety levels of individuals who can lose their jobs with the development of AI techniques and products. Sociotechnical blindness dimension is used to measure anxiety levels of individuals who cannot properly realize that AI can only work with the combination of people and social institutions (Johnson & Verdicchio, 2017). This is a misunderstanding concept that AI technology in the future can operate per se without the involvement of human beings. Last, AI configuration dimension similar to robot-anxiety construct can be attributed to anxiety levels of individuals who may think humanoid AI techniques/products are scary and intimidating (Wang & Wang, 2019). 1.1. Artificial Intelligence in Education Similar to other fields, technological advancements in education have continued to evolve within the last decades. Since the development of microcomputers to personal computers in the 1970s, applications of information and computer-related technologies have been recently increased in various ways in education. AI in education, for instance, can be used in computer aided instruction, global learning, individualized learning, adaptive learning, and enhanced efficiency and effectiveness in educational administration among many other examples (Chen, Chen, & Lin, 2020; Timms, 2016). The main purpose of using AI in education is to enhance the learning experiences of students in effective and efficient ways. In doing so, cobots, the application of robots helping teachers in a classroom, are being used to adjust learning environments according to students’ skills (Timms, 2016). Furthermore, intelligent tutoring systems have different functions that can be carried out for prompt feedback on students’ learning experiences and assignments (Mikropoulos & Natsis, 2011; Rus, D’Mello, Hu, & Graesser, 2013). It also offers powerful pedagogical tools that can help enhance instructional quality (Chen et al., 2020). These tools such as simulation-based instructions including various technologies (e.g., virtual reality and 3-D technology) can help students have practical and experimental learning experiences (Mikropoulos & Natsis, 2011; Timms, 2016; Wartman & Combs, 2018). Although it is still inevitable that computerization and automation can take over human work, teachers still play the main role in education. However, teachers and other educational stakeholders can need to International Online Journal of Education and Teaching (IOJET) 2020, 7(4), 1501-1515. 1503 adapt AI technologies and products so that they can get timely prepared to fulfill these types of AI developments in education. This study has two purposes. The main goal was to adapt the Artificial Intelligence Anxiety Scale (AIAS) into Turkish and investigate validity and reliability properties of the scale. In the second part, it was further aimed to explore whether the AIA levels of teachers differ based on gender, degree levels, age of teachers, and years of experiences in teaching. 2. Method In this part of the study, information about participants, the data collection instrument, the steps of the scale adaptation, data collection procedure, and data analyses was provided. 2.1. Participants The data were collected from teachers who were teaching from primary through high school-level students in the academic year of 2019-2020 in Turkey. The purpose of the study was shared with participants who were asked to voluntarily involve in the study by filling out the items through an online survey form. Since each item was required to respond to the next following item, there were no missing data. The distribution of 222 teachers is as follows: 49.1% (N = 109) male and 50.9% (N = 113) female; 79.7% (N = 177) an undergraduate degree, 7.7% (N = 17) a master’s degree without thesis, 9.9% (N = 22) a master’s degree with thesis, and 2.7% (N = 6) a doctoral degree; the mean of the teachers’ age was 33.6, ranging from 22 to 57 years old; the average year of teachers’ teaching experience was 9.7, ranging from 1 to 36 years of teaching. 2.2. Data Collection Instrument The data were obtained using the Artificial Intelligence Anxiety Scale (AIAS; Wang & Wang, 2019), which was adapted into Turkish by the author in this study. Wang and Wang (2019) first adapted 59 items based on numerous studies relevant to AIA. Those 59 items were revised by experts and 9 items were eliminated from the scale because of redundancy. Psychometric properties of the scale were investigated after administering the remaining 50 items to 301 participants. As a result of exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) as well as reliability analyses, 21 items with four dimensions (i.e., learning, job replacement, sociotechnical blindness, and AI configuration) were retained, which is reported in Table 1. As implemented in the original scale, the response scale of items was based on a 7-point Likert-type (1 = never through 7 = completely). In the original study, Wang and Wang (2019) reported the reliability of each dimension; .974 for learning (L), .917 for job replacement (JR), .917 for sociotechnical blindness (SB), and .916 for AI configuration (AIC). Furthermore, corrected item-to-total correlation for each item was higher than .40, which was above the critical value of .30 (Nurosis, 1994). Table 1. Dimensions of AIAS and corresponding items Dimensions Items Learning (8 items) 1-8 Job Replacement (6 items) 9-14 Sociotechnical Blindness (4 items) 15-18 AI Configuration (3 items) 19-21 Terzi 1504 2.3. The Adaptation of the AIAS Since the main purpose of this study was to adapt the AIAS into Turkish, items in the original scale were first translated by three academicians who have obtained their Ph.D. degrees from English-spoken countries. Second, the consistency of the translated version of the scale was analyzed and reconciled with the translations. Third, the Turkish version of the scale was then translated back into English by another academician who holds a Ph.D. degree from an English-spoken country, which was then compared with the original items. The original and translated items of the AIAS can be seen in Appendix 1. 2.4. Data Collection Procedure Data were collected through an online survey form. The link of the survey was send to in- service teachers via email and social media platforms. Given the first item asking participants if they would like to voluntarily attend the study, 222 teachers were responded to 11 items about socio-demographic characteristics (e.g., gender, degree levels, age, and years of teaching experience) of participants and 21 items about AIA. Gender and degree levels were treated as categorical variables; whereas, age and years of teaching experience were treated as continuous variables. The “convenience sampling” method was considered for this study. Since the sample size around 10 times the number of items was acceptable (Kline, 2015; Nunnally, 1978), the data collected from 222 teachers suffices for the analyses with the AIAS of 21 items. 2.5. Data Analysis First of all, no missing data were observed because of the fact that participants cannot proceed to the next following item unless a previous item was responded. Next, data were further investigated based on skewness, kurtosis, and outliers. Moreover, for the sake of the construct validity, the data collected using the AIAS was carried out based on confirmatory factor analysis (CFA) using Mplus 7.0 version (Muthén & Muthén, 2012). Nevertheless, the correlation among the sub-dimensions of the scale and the entire scale as well as the reliability coefficients of the sub-dimensions of the scale and the entire scale in terms of internal consistency were calculated using SPSS version 22.0 (IBM Corp, 2013). Furthermore, after ensuring validity and reliability of the AIAS, anxiety levels of teachers based on gender, degree levels, age of teachers, and years of teaching experience were investigated. In doing so, anxiety levels of teachers based on gender and degree levels were compared by independent sample t- test and ANOVA, respectively. Furthermore, whether anxiety levels of teachers are correlated with teachers’ age and years of teaching experience was investigated by Pearson correlation coefficients. 3. Results In this part of the study, the validity and reliability properties of the adapted version of the AIAS were reported. Additionally, after ensuring validity and reliability of the scale, AIA levels of teachers based on socio-demographic factors were provided. 3.1. Validity and Reliability Analyses CFA was implemented to check how well latent construct can be explained by items of the AIAS (Suhr, 2006) given the dimensions of learning (L), job replacement (JR), sociotechnical blindness (SB), and AI configuration (AIC). The diagram for CFA was presented in Figure 1. Although the chi-square value was found significant (χ2 = 458.268, df = 178, p < 0.05) given the large number of degrees of freedom, χ2 / df = 2.57 is within the acceptable level between 2 and 3 (Bentler & Hu, 1995). Moreover, other indices such as the Tucker-Lewis index (TLI; Tucker & Lewis, 1973), the comparative fit index (CFI; Bentler, 1990), the root mean square error of approximation (RMSEA), and the standardized root mean square residual (SRMR) International Online Journal of Education and Teaching (IOJET) 2020, 7(4), 1501-1515. 1505 were investigated. While the TLI, CFI, and SRMR values of .93, .94, and .069, respectively, showed a good fit of the model to the data, the RMSEA value of .084 displayed an acceptable fit (Bentler & Hu, 1995; Browne & Cudeck, 1993; Kline, 2015). Figure 1. CFA Diagram of the AIAS Nevertheless, the Cronbach’s Alpha reliability coefficient was .96 for the complete scale (21 items), .89 for L (8 items), .95 for JR (6 items), .89 for SB (4 items), and .95 for AIC (3 items) dimensions. As a result, strong evidence of consistency in teachers’ responses to the AIAS items was observed. Table 2. Means, Standard Deviations, and Correlations of AIAS total score and its sub- dimensions X̅ SD Correlation L JR SB AIC L 1.98 1.036 JR 3.47 1.879 0.567 SB 3.95 1.805 0.442 0.813 AIC 3.34 1.959 0.554 0.819 0.771 AIAS 3.18 1.475 0.675 0.939 0.899 0.926 Note. Correlations are significant at the 0.01 level (2-tailed); N = 222; L = Learning; JR = Job Replacement; SB = Sociotechnical Blindness; AIC = AI Configuration; AIAS = Artificial Intelligence Anxiety Scale. Table 2 reports the mean, standard deviation, and correlations of AIAS total score and its sub-dimensions. The mean values were relatively low based on the 7-point Likert-type scale, ranging from 1.98 (SD = 1.036) for L to 3.95 (SD = 1.805) for SB. The correlations among the sub-dimensions were positive and statistically significant at the 0.001 level, which ranged from Terzi 1506 0.442 between L and SB to 0.819 between JR and AIC. These findings show that the scale has a structure compatible with its sub-dimensions. Furthermore, the average score and standard deviations of each item as well as correlations among items were reported in Appendix 2. Based on the aforementioned findings, anxiety levels of teachers based on gender and degree they hold were compared by independent sample t-test and ANOVA, respectively. Furthermore, whether anxiety levels of teachers are correlated with teachers’ age and years of teaching experience was explored. Given the medium-sized sample of 222, we retain the null hypothesis of the distribution of normal sample because absolute z-values of skewness and kurtosis are below 3.29 with an alpha level of 0.05 (West, Finch, & Curran, 1995). Thus, we conclude that parametric tests can be used. Moreover, homogeneity of variance (i.e., homoscedasticity) was checked based on Levene’s test for ANOVA. The null hypothesis of the equality of variances for variables based on each sub-dimension and the total scale was accepted; therefore, we concluded that the population variances are equal across groups. Table 3. Artificial Anxiety Level Comparisons for Each Dimension Learning �̅� SD p Skewness Kurtosis Gender Male 1.76 0.926 0.002 1.196 .806 Female 2.18 1.098 Job Replacement �̅� SD p Skewness Kurtosis Male 3.12 1.779 0.007 .302 -1.096 Female 3.80 1.919 Sociotechnical Blindness �̅� SD p Skewness Kurtosis Male 3.74 1.802 0.084 -.026 -1.087 Female 4.16 1.791 AI Configuration �̅� SD p Skewness Kurtosis Male 3.02 1.986 0.019 .314 -1.167 Female 3.64 1.892 AIAS �̅� SD p Skewness Kurtosis Male 2.91 1.418 0.007 .230 -1.023 Female 3.44 1.487 Note. Values of standard error of skewness and kurtosis were .163 and .325, respectively. Table 3 shows the comparisons of AIA levels of teachers based on gender for each dimension and the entire scale. For sociotechnical blindness dimension, male and female International Online Journal of Education and Teaching (IOJET) 2020, 7(4), 1501-1515. 1507 teachers did not show any significant differences (�̅�𝑀 = 3.74, �̅�𝐹 = 4.16; p = .084). However, for learning (�̅�𝑀 = 1.76, �̅�𝐹 = 2.18; p = .002), job replacement (�̅�𝑀 = 3.12, �̅�𝐹 = 3.80; p = .007), AI configuration (�̅�𝑀 = 3.02, �̅�𝐹 = 3.64; p = .019) dimensions and the overall AIA scale (�̅�𝑀 = 2.91, �̅�𝐹 = 3.44; p = .007), female teachers were significantly more anxious towards AI than male teachers. Furthermore, there were no differences in anxiety observed towards AI based on degree levels teachers hold. In other words, ANOVA results showed that teachers’ anxiety levels were not different across degree levels (i.e., undergraduate, master without thesis, master with thesis, and doctorate): F(3, 218) = .37, p = .773 for learning; F(3, 218) = .75, p = .523 for job replacement; F(3, 218) = 1.70, p = .168 for sociotechnical blindness; F(3, 218) = .62, p = .603 for AI configuration; and F(3, 218) = .97, p = .408 for the AIA total score. Moreover, whether anxiety levels of teachers were correlated with ages and years of teaching experience was carried out by Pearson correlation coefficients based on each dimension and overall scale of AIA. Results reported in Table 4 showed that the anxiety levels of teachers are not correlated with age and years of experience (i.e., p > .05). That is, the anxiety levels of teachers based on AIAS total score and its sub-dimensions did not differ in any directions across ages and years of teaching experience. Table 4. Pearson Correlations L JR SB AIC AIAS Age R .099 -.003 .073 .036 .051 p .143 .969 .278 .596 .450 Experience R .120 .003 .077 .034 .057 p .075 .965 .253 .616 .398 Note. R is the Pearson Correlation; p is the significance level. 4. Discussion and Conclusion Artificial intelligence (AI) is human-like automation that can operate many functions based on some level of intelligence (Coppin, 2004), which brings many advantages to various sectors (e.g., healthcare, engineering, finance, agriculture, law, and education). AI technology has been also used in the classroom for learning purposes (Luckin, Holmes, Griffiths, & Forcier, 2016). For instance, computer aided instruction, personalized learning, and enhanced efficiency and effectiveness in educational administration are among many other applications of AI in education (Chen et al., 2020; Timms, 2016). The main purpose of using AI in education is to enhance the learning experiences of students in effective and efficient ways. Although it is still inevitable that AI technology can take over human work (Wang & Siau, 2019), teachers have irreplaceable roles in education. However, teachers and other educational stakeholders still should be timely prepared for AI developments in education. Nevertheless, this situation makes individuals nervous and anxious because they need to adjust themselves to the changing world with AI technologies. AI anxiety (AIA) can be defined as the panic and nervousness due to unknown directions of AI technologies and products (Johnson & Verdicchio, 2017). Despite varying anxiety levels among people, Wang and Wang (2019) recently developed an AIA scale with four sub-dimensions; learning, job replacement, sociotechnical blindness, and AI configuration. Therefore, the main purpose of this study was Terzi 1508 to adapt the Artificial Intelligence Anxiety Scale (AIAS) into Turkish and analyze its validity and reliability properties based on data collected from in-service teachers. The adapted version of the AIAS was investigated for the construct validity based on CFA that provided a good fit of the model to the data. Moreover, the reliability coefficients of the dimensions of the AIAS in terms of internal consistency showed strong evidence of consistency in teachers’ responses to the AIAS items. Furthermore, after ensuring validity and reliability of the scale, anxiety levels of teachers based on gender, degree levels, age, and years of teaching experience were compared in terms of each sub-dimension and the total AIAS score. For sociotechnical blindness dimension, male and female teachers did not show any significant differences. It is important to note that both in-service and prospective teachers should acquire fundamental skills and knowledge in technology for effective teaching (Hofer & Swan, 2008) regardless of gender differences. For instance, Terzi stated that when prospective teachers have higher competency in techno- pedagogy, they can build more effective learning atmosphere for students because of the fact that they believe they can properly design the instructional process (2020). However, for learning, job replacement, AI configuration dimensions and the overall AIA scale, female teachers were significantly more anxious towards AI than male teachers. These findings suggest that teachers, especially female teachers, need training with AI technologies and products so that they can feel more confident to adapt themselves to the changing requirements of the age. It is a crucial duity for teachers to prepare students with the strength and power of AI that can let them disclose and develop their abilities in this changing labor (Luckin et al., 2016). Moreover, there were no differences observed in anxiety levels of teachers towards AI based on degree levels they hold. Additionally, anxiety levels of teachers were not related to teachers’ age and years of experience in teaching. Thus, regardless of degree levels, teachers’ age, and years of teaching experience, teachers should be provided with appropriate training on AI technologies to be used in a classroom. For instance, the application of robots (i.e., cobots) can be applied to adjust learning environments based on students’ skills (Timms, 2016). Furthermore, intelligent tutoring systems can be carried out for prompt feedback on students’ learning experiences and assignments (Mikropoulos & Natsis, 2011; Rus, D’Mello, Hu, & Graesser, 2013). Powerful pedagogical tools such as simulation-based instructions including various technologies (e.g., virtual reality and 3-D technology) can also offer practical and experimental learning experiences (Chen et al., 2020; Mikropoulos & Natsis, 2011; Timms, 2016; Wartman & Combs, 2018). It is crucial to note that although computerization and automation can somewhat take over human work, teachers still play the main role in education. However, teachers and other educational stakeholders still need to adapt AI technologies and products so that they can get timely prepared to fulfill these types of AI developments in education. There were some limitations with this study. First, since the scale was originally developed to measure the general public anxiety toward AI development, it would be necessary to develop a new scale to measure anxiety levels of teachers. Second, the data using the AIAS were obtained based on the self-assessment of teachers that can be potentially affected by their subjective ideas and perceptions. Thus, the results of this study should be interpreted with caution. In the future, findings based on the data obtained from the AIAS can be further verified with more teachers as well as pre-service teachers. 5. Conflict of Interest The author confirms that there is no conflict of interest. International Online Journal of Education and Teaching (IOJET) 2020, 7(4), 1501-1515. 1509 6. Ethics Committee Approval Ethics committee approval was received from Harran University (No: 76244175-E.20907) Terzi 1510 References Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238-246. Bentler, P., M., & Hu. P. (1995). EQS: Structural equations program manual. Los Angeles, CA: BMPD Statistical Software. Bostrom, N. (2014). Superintelligence: Paths, dangers, strategies. Oxford, UK: Oxford University Press. Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–162), Thousand Oaks, CA: Sage. Cairns, L., & Malloch, M. (2017). Computers in education: The impact on schools and Classrooms. In R. Maclean (Ed.), Life in schools classrooms: Past, present and future (pp. 603-617). Singapore: Springer Nature. Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review. IEEE Access, 8, 75264-75278. Chou, C. (2003). Incidences and correlates of internet anxiety among high school teachers in Taiwan. Computers in Human Behavior, 19(6), 731–749. Chuo, Y. H., Tsai, C. H., Lan, Y. L., & Tsai, C. S. (2011). The effect of organizational support, self efficacy, and computer anxiety on the usage intention of e-learning system in hospital. African Journal of Business Management, 5(14), 5518–5523. Coppin, B. (2004). Artificial intelligence illuminated. Boston, MA: Jones & Bartlett Learning. Esterhuyse, M. P., Scholtz, B. M., & Venter, D. (2016). Intention to use and satisfaction of e- learning for training in the corporate context. Interdisciplinary Journal of Information, Knowledge, and Management, 11, 347–365. Future of Life Institute (FLI). (2015). Autonomous weapons: An open letter from AI & robotics researchers. Retrieved from http://futureoflife.org/open-letter-autonomous-weapons/ Guilford, J. P. (1954). Psychometric methods (2nd Ed.). New York: McGraw-Hill Book Corp. Hofer, M., & Swan, K. O. (2008). Technological pedagogical content knowledge in action: A case study of a middle school digital documentary project. Journal of Research on Technology in Education, 41(2), 179-200. IBM Corp. (2013). IBM SPSS statistics for Windows (Version 22.0). Armonk, NY: IBM Corp. Johnson, D. G., & Verdicchio, M. (2017). AI anxiety. Journal of the Association for Information Science and Technology, 68(9), 2267-2270. Kline, R. B. (2015). Principles and practice of structural equation modeling (4th Edition). New York: Guilford Press. Korobili, S., Togia, A., & Malliari, A. (2010). Computer anxiety and attitudes among undergraduate students in Greece. Computers in Human Behavior, 26(3), 399–405. Luckin, R., Holmes, W., Griffiths, M., & Forcier, L. B. (2016). Intelligence unleashed. An argument for AI in education. Pearson. Manyika, J., Lund, S., Chui, M., Bughin, J., Woetzel, J., Batra, P., …Sanghvi, S. (2017). Jobs lost, jobs gained: Workforce transitions in a time of automation. San Francisco, CA: McKinsey Global Institute. http://futureoflife.org/open-letter-autonomous-weapons/ International Online Journal of Education and Teaching (IOJET) 2020, 7(4), 1501-1515. 1511 Marcoulides, G. A. (1989). Measuring computer anxiety: The computer anxiety scale. Educational and Psychological Measurement, 49, 733-739. Mikropoulos, T. A., & Natsis, A. (2011). Educational virtual environments: A ten-year review of empirical research (1999–2009). Computers & Education, 56(3), 769-780. Muthén, L. K., & Muthén, B. O. (2012). Mplus user's guide (7th Edition), Los Angeles, CA: Muthén & Muthén. Nagao K. (2019) Artificial intelligence in education. In K. Nagao (Ed.), Artificial intelligence accelerates human learning (pp. 1-17). Springer, Singapore. Nauman, Z. (2017, February 17). AI will make life meaningless, Elon Musk warns. Retrieved on Jun 3, 2020 from https://nypost.com/2017/02/17/elon-musk-thinks-artificial- intelligence-will-destroy-the-meaning-of- life/?utm_campaign=SocialFlow&utm_source=NYPFacebook&utm_medium=Social Flow&sr_share=facebook Nurosis, M. (1994). Statistical data analysis. Chicago, IL: SPSS Inc. Nomura, T., Suzuki, T., Kanda, T., & Kato, K. (2006, September). Measurement of anxiety toward robots. In Robot and Human Interactive Communication. The 15th IEEE International Symposium on Robot and Human Interactive Communication (pp. 372– 377). Nunnally, J. C., & Bernstein, I. H. (1978). Psychometric theory. New York: McGraw-Hill. Rus, V., D’Mello, S., Hu, X., & Graesser, A. (2013). Recent advances in conversational intelligent tutoring systems. AI Magazine, 34(3), 42-54. Suhr, D. (2006). Exploratory or confirmatory factor analysis. SAS Users Group International Conference (pp. 1 - 17). Cary: SAS Institute, Inc. Retrieved from http://www2.sas.com/proceedings/sugi31/200-31.pdf Terzi, R., (2020). The impact of understanding learners and techno-pedagogical competency on effective learning environments by designing the instructional process. Turkish Journal of Education, 9(3), 246-259. Tucker, L. R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1-10. Timms, M. J. (2016). Letting artificial intelligence in education out of the box: Educational cobots and smart classrooms. International Journal of Artificial Intelligence in Education, 26(2), 701-712. Wang, Y. S. (2007). Development and validation of a mobile computer anxiety scale. British Journal of Educational Technology, 38(6), 990–1009. Wang, W., & Siau, K. (2019). Artificial intelligence, machine learning, automation, robotics, future of work and future of humanity: A review and research agenda. Journal of Database Management, 30(1), 61-79. Wang, Y. Y., & Wang, Y. S. (2019). Development and validation of an artificial intelligence anxiety scale: An initial application in predicting motivated learning behavior. Interactive Learning Environments, 1-16. Wartman, S. A., & Combs, C. D. (2018). Medical education must move from the information age to the age of artificial intelligence. Academic Medicine, 93(8), 1107-1109. https://nypost.com/2017/02/17/elon-musk-thinks-artificial-intelligence-will-destroy-the-meaning-of-life/?utm_campaign=SocialFlow&utm_source=NYPFacebook&utm_medium=SocialFlow&sr_share=facebook https://nypost.com/2017/02/17/elon-musk-thinks-artificial-intelligence-will-destroy-the-meaning-of-life/?utm_campaign=SocialFlow&utm_source=NYPFacebook&utm_medium=SocialFlow&sr_share=facebook https://nypost.com/2017/02/17/elon-musk-thinks-artificial-intelligence-will-destroy-the-meaning-of-life/?utm_campaign=SocialFlow&utm_source=NYPFacebook&utm_medium=SocialFlow&sr_share=facebook https://nypost.com/2017/02/17/elon-musk-thinks-artificial-intelligence-will-destroy-the-meaning-of-life/?utm_campaign=SocialFlow&utm_source=NYPFacebook&utm_medium=SocialFlow&sr_share=facebook http://www2.sas.com/proceedings/sugi31/200-31.pdf Terzi 1512 West, S. G., Finch, J. F., & Curran, P. J. (1995). Structural equation models with nonnormal variables: problems and remedies. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues and applications (pp. 56-75). Newbery Park, CA: Sage. Wu, Y. H., Wrobel, J., Cornuet, M., Kerhervé, H., Damnée, S., & Rigaud, A. S. (2014). Acceptance of an assistive robot in older adults: A mixed-method study of human– robot interaction over a 1-month period in the Living Lab setting. Clinical Interventions in Aging, 9, 801-811. International Online Journal of Education and Teaching (IOJET) 2020, 7(4), 1501-1515. 1513 Appendix 1. Orginal Items Translated Items (Turkish) L e a rn in g 1 Learning to understand all of the special functions associated with an AI technique/product makes me anxious. Bir YZ tekniğiyle/ürünüyle ilişkili tüm özel işlevleri anlamayı öğrenmek beni endişelendiriyor. 2 Learning to use AI techniques/products makes me anxious. YZ tekniklerini/ürünlerini kullanmayı öğrenmek beni endişelendiriyor. 3 Learning to use specific functions of an AI technique/product makes me anxious. Bir YZ tekniğinin/ürününün belirli işlevlerini kullanmayı öğrenmek beni endişelendiriyor. 4 Learning how an AI technique/product works makes me anxious. Bir YZ tekniğinin nasıl çalıştığını (veya ürününün ne işe yaradığını) öğrenmek beni endişelendiriyor. 5 Learning to interact with an AI technique/product makes me anxious. Bir YZ tekniği/ürünü ile etkileşim kurmayı öğrenmek beni endişelendiriyor. 6 Taking a class about the development of AI techniques/products makes me anxious. YZ tekniklerinin/ürünlerinin geliştirilmesi hakkında ders almak beni endişelendiriyor. 7 Reading an AI technique/product manual makes me anxious. Bir YZ tekniğinin/ürününün kılavuzunu okumak beni endişelendiriyor. 8 Being unable to keep up with the advances associated with AI techniques/products makes me anxious. YZ teknikleriyle/ürünleriyle ilişkili gelişmelere ayak uyduramamak beni endişelendiriyor. 9 I am afraid that an AI technique/product may make us dependent. Bir YZ tekniğinin/ürününün bizi bağımlı kılabileceğinden korkuyorum. Jo b R e p la c e m e n t 10 I am afraid that an AI technique/product may make us even lazier. Bir YZ tekniğinin/ürününün bizi daha da tembelleştirebileceğinden korkuyorum. 11 I am afraid that an AI technique/product may replace humans. Bir YZ tekniğinin/ürününün insanların yerini alabileceğinden korkuyorum. 12 I am afraid that widespread use of humanoid robots will take jobs away from people. İnsansı robotların yaygın kullanımının, insanların işlerini elinden alacağından korkuyorum. 13 I am afraid that if I begin to use AI techniques/products I will become dependent upon them and lose some of my reasoning skills. YZ tekniklerini/ürünlerini kullanmaya başlarsam onlara bağımlı olacağımdan ve akıl yürütme becerilerimi kaybedeceğimden korkuyorum. 14 I am afraid that AI techniques/products will replace someone’s job. YZ tekniklerinin/ürünlerinin kişilerin işlerini elinden alacağından korkuyorum. Terzi 1514 Appendix 1 (Cont.). Orginal Items Translated Items (Turkish) S o c io te c h n ic a l B li n d n e ss 15 I am afraid that an AI technique/product may be misused. Bir YZ tekniğinin/ürününün kötü amaçlı kullanılabileceğinden korkuyorum. 16 I am afraid of various problems potentially associated with an AI technique/product. Bir YZ tekniğiyle/ürünüyle potansiyel olarak ilişkili çeşitli sorunlardan korkuyorum. 17 I am afraid that an AI technique/product may get out of control and malfunction. Bir YZ tekniğinin/ürününün kontrolden çıkabilir ve arızalanabilir olacağından korkuyorum. 18 I am afraid that an AI technique/product may lead to robot autonomy. Bir YZ tekniğinin/ürününün robot özerkliğine yol açabileceğinden korkuyorum. A I C o n fi g u ra ti o n 19 I find humanoid AI techniques/products (e.g. humanoid robots) scary. İnsansı YZ tekniklerini/ürünlerini (örneğin insansı robotları) ürkütücü buluyorum. 20 I find humanoid AI techniques/products (e.g. humanoid robots) intimidating. İnsansı YZ tekniklerini/ürünlerini (örneğin insansı robotları) tehditkar buluyorum. 21 I don’t know why, but humanoid AI techniques/products (e.g. humanoid robots) scare me. Nedenini bilmiyorum, fakat insansı YZ teknikler/ürünler (örneğin insansı robotlar) beni korkutuyor. Appendix 2. Means, Standard Deviations, and Correlations of each item �̅� SD i1 i2 i3 i4 i5 i6 i7 i8 i9 i10 i11 i12 i13 i14 i15 i16 i17 i18 i19 i20 i1 2.04 1.44 i2 1.81 1.24 .733** i3 1.80 1.26 .608** .762** i4 1.76 1.26 .615** .750** .823** i5 1.88 1.28 .674** .795** .783** .831** i6 1.61 1.12 .532** .715** .647** .592** .708** i7 1.66 1.14 .475** .582** .548** .511** .577** .698** i8 3.24 2.03 .255** .293** .321** .282** .289** .225** .268** i9 3.10 2.05 .425** .381** .421** .354** .477** .363** .312** .464** i10 3.56 2.14 .414** .375** .381** .347** .448** .309** .296** .425** .826** i11 3.45 2.16 .421** .434** .452** .409** .495** .352** .307** .353** .711** .778** i12 3.87 2.13 .366** .378** .367** .327** .424** .271** .291** .369** .689** .742** .842** i13 3.09 2.09 .483** .451** .476** .439** .524** .409** .341** .382** .732** .710** .754** .731** i14 3.72 2.10 .348** .369** .387** .351** .403** .273** .268** .367** .654** .655** .813** .863** .756** i15 4.83 2.08 .182** .185** .185** .162* .221** .112 .119 .453** .549** .613** .565** .647** .490** .626** i16 3.82 1.98 .344** .329** .337** .296** .400** .273** .215** .410** .645** .650** .662** .679** .664** .668** .751** i17 3.71 2.08 .316** .311** .289** .213** .317** .289** .283** .357** .606** .639** .608** .643** .625** .579** .645** .783** i18 3.44 2.14 .321** .336** .313** .261** .378** .328** .306** .353** .624** .609** .694** .671** .707** .662** .504** .669** .727** i19 3.35 2.05 .407** .455** .456** .431** .512** .341** .370** .367** .603** .661** .737** .734** .680** .668** .560** .626** .611** .731** i20 3.42 2.07 .385** .411** .402** .380** .475** .326** .348** .339** .636** .665** .734** .742** .668** .694** .558** .626** .634** .773** .884** i21 3.24 2.04 .438** .490** .447** .398** .497** .422** .363** .362** .657** .663** .762** .741** .747** .727** .515** .619** .644** .782** .859** .856** Notes. ** indicates significant correlations at the 0.01 level (2-tailed); * indicates significant correlations at the 0.05 level (2-tailed).