Ataberk, B. Mirici, İ. H. (2022). An investigation of the 21st century skills in English language teaching (ELT) programs in turkey. International Online Journal of Education and Teaching (IOJET), 9(4). 1513-1544. Received : 10.05.2022 Revised version received : 27.07.2022 Accepted : 01.08.2022 AN INVESTIGATION OF THE 21ST CENTURY SKILLS IN ENGLISH LANGUAGE TEACHING (ELT) PROGRAMS IN TURKEY (Research article) Correspondence Berna Ataberk Hacettepe University, Turkey bernataberk@gmail.com İsmail Hakkı Mirici - https://orcid.org/0000-0002-0906-0259 Hacettepe University, Turkey hakkimirici@gmail.com Biodata(s): Berna Ataberk is a Research Assistant at Hacettepe University Faculty of Education, Department of English Language Teaching. She has completed her MA thesis at Graduate School of Educational Sciences in 2022. İsmail Hakki Mirici is a full time Professor at Hacettepe University Faculty of Education, Department of English Language Teaching. His research interests are EFL Teacher Education, Curriculum Development and Foreign Language Education Policy in Europe. Copyright © 2014 by International Online Journal of Education and Teaching (IOJET). ISSN: 2148-225X. Material published and so copyrighted may not be published elsewhere without written permission of IOJET. mailto:bernataberk@gmail.com https://orcid.org/0000-0002-0906-0259 mailto:hakkimirici@gmail.com http://orcid.org/xxxx Ataberk & Mirici 1514 AN INVESTIGATION OF THE 21ST CENTURY SKILLS IN ENGLISH LANGUAGE TEACHING (ELT) PROGRAMS IN TURKEY Berna Ataberk bernataberk@gmail.com İsmail Hakkı Mirici hakkimirici@gmail.com Abstract The rapidly developing technology has affected our lives in many ways. The Internet has now directly affected our daily and business life, our education system and even our social life. For this reason, there is a need to develop some new skills required by our age in education. The concepts such as “Life and Career Skills”, “Learning and Innovation Skills”, “Information, Media and Technology Skills” have also entered our lives as the 21st century skills. This study aimed to investigate to what extent the 21st century skills were included in English language teacher training programs; to compare the perspectives of students and academics of the related programs with the program and to discuss the effectiveness of the program in terms of developing the 21st century skills. For this purpose, the study adopted mixed methods research design based on both qualitative and quantitative data. The data were collected using triangulation method via document analysis, questionnaires for students and academics and face to face interviews. In addition, the ELT department programs of randomly selected Faculties of Education of 4 different universities were examined. The results have revealed that the curriculum provided by the Council of Higher Education are applied in all the participating ELT departments, and it aims to develop the 21st century skills in prospective EFL teachers at some points. However, the course contents of the curricula in those departments are limited to some of the skills which may break the integrity of the 21st century skills in professional life reflections. The participant academics and students believe that some of the components of the 21st century skills are included in their course contents but not completely. The students do not feel confident about handling the 21st century skills in a real-life implementation. Keywords: 21st century skills; English Language teacher education; effectiveness of the curricula; ELT departments 1. Introduction Teacher training has an essential role for every country since educational outputs are important factors in shaping the long-term achievements of a nation. Therefore, qualified and well-designed teacher training curricula is of great importance. According to Garcia (2012), teachers learning plays a crucial role in developing 21st century skills, such as problem solving collaboratively, using technological devices to build their knowledge base, and being involved in reflection and action. Upon the introduction of the concept of 21st century skills, the teacher training curricula have been designed accordingly at a global scale in order to bring up teachers who can think critically, collaborate, communicate and solve sudden problems. Redmond (2016) has defined 21st century skills as “the knowledge, skills, and expertise necessary for individuals to fully participate in the digital world and global community of the 21st century”. The teachers of the 21st century should be able to cope with the rapid changes around them, and this can only be possible through updating their knowledge and skills in accordance with the rapidly changing economic, political, and social mailto:bernataberk@gmail.com mailto:hakkimirici@gmail.com International Online Journal of Education and Teaching (IOJET) 2022, 9(4), 1513-1544. 1515 conditions and changing realities of a globalized society (Darling-Hammond, 2006; Garrett‐ Rucks, 2016; Geisler et al., 2007; Moeller & Abbott, 2018; National Research Council, 2012; Quicios, 2018; Csikszentmihalyi & Schneider, 2000; The National Standards Collaborative Board, 2015; Tochon, 2009, Cox & Montgomery, 2019; Karani & Mary, 2022). In Turkey, the curricula of all faculties of education and English Language Teaching (ELT) departments have been designed by the Council of Higher Education, Turkey (CoHE) which was established in 1981. The graduates of ELT programs in Turkey are expected to develop the following skills (CoHE, 2018). a) use the language and skills efficiently; b) master at teaching methods and techniques of a language; c) be informed in general culture subjects as a teacher; d) have a command of education sciences principles; e) have the background knowledge in psychological, sociological, philosophical, anthropological aspects of education; f) be social, interactive, innovative, conscious, and open-minded as teachers; g) tolerate the differences among individuals in-school and out of school environments; h) be able to prepare lessons in accordance with the technological developments. This study is conducted to investigate the effectiveness of the English language teacher training programs in Turkey in terms of developing 21st century skills in the prospective EFL teachers. 2. Review of literature The term ‘21st century skills’ refer to the 3 core competencies namely; reading, writing and arithmetic, as well as many other essential skills, such as digital literacy, critical thinking, communication, collaboration, global competencies, and problem-solving skills (Yıldız & Parlak, 2016). Figure 1, below, shows the components of the 21st century skills. Figure 1. Framework for 21st Century Learning (P21 Framework, 2019a) As is seen in the figure, Life and Career Skills include 1) flexibility and adaptability, 2) initiative and self-direction, 3) social and cross-cultural skills, 4) productivity and accountability, 5) leadership and responsibility. Learning and Innovation skills have sub- categories named as 4Cs which are 1) critical thinking, 2) communication, 3) collaboration Ataberk & Mirici 1516 and 4) creativity. Information, Media, and Technology skills incorporate 1) information literacy, 2) media literacy, 3) ICT (information, communications, and technology) literacy (P21, 2019a). The new generation brings about new needs and educational principles which include use of technology, particular ways of thinking such as creativity, innovation, critical thinking etc. (Patrick and Esther, 2015). In order to enhance these particular thinking ways, HOTS are needed to develop in learners. As one of the essential skills for the students to adapt the globalizing world, HOTS should be integrated in all fields of education (Soulé and Warrick, 2015; Tyas, Nurkamto & Marmanto, 2020). According to Lewis (1993), HOTS activities can engage any student, they do not specify a group of students; any activity may require HOTS, and this depends on the intellectual history of the teacher. Teachers’ intellectuality is a determining point here since they also need to have the skills to design and conduct classroom activities effectively, they need to be able to communicate well and use technology efficiently and constantly improve themselves professionally (Darling-Hammond, 2006). Teacher training programs have a specific importance in that prospective teachers should know how to address unexpected problems and learning needs of their future students. This suggests that curricula must help prospective teachers develop ‘the disposition to continue to seek answers to difficult problems of teaching and learning’ (Darling-Hammond, 2006). Advancing circumstances made it necessary for policymakers to develop education systems based on increasing students’ higher order thinking skills as critical and creative thinking, collaborate and communicate well (4Cs) (Bedir, 2019). The higher order thinking skills (HOTS) is also essential since it is one of the elemental parts of 21st century skills for student-teachers to face the global challenge. (Soulé & Warrick, 2015). The changes have also been visible in teacher education and ‘teachers are expected to be problem solvers and capable of recognizing and weighing diverse perspectives’ (Sprott, 2019; Bedir, 2019). According to the Council of Higher Education’s booklet in 2018, the English Language Teaching program consists of 240 ECTS and 155 lecture hours. 48% (71 hours) of the program contains departmental courses, 34% (56 hours) of it contains educational sciences courses and 18% (28 hours) contains general knowledge courses (CoHE, 2018). Each university and each department may offer various courses for general culture elective courses in addition to the ones that are stated in the booklet of Council of Higher Education (2018). In addition to the 3 categories of courses, in majority universities, teacher candidates are expected to take the Orientation to Campus Life course in the first semester. In this study, it is aimed to answer the following research questions: Main research question: To what extent are the 21st century skills included in the curricula of English Language Teaching programs in Turkey? Sub-research questions: 1.What grades of ELT departments in the participating institutions have courses aiming to develop the 21st century skills? 2.What are the components of the courses aiming to develop the 21st century skills in the participating ELT departments? 3. How much space is given to the teaching of the 21st century skills the participating ELT departments? International Online Journal of Education and Teaching (IOJET) 2022, 9(4), 1513-1544. 1517 4.What do participant student teachers of English think about developing the 21st century skills in their department? 5. What do participating academics think about developing the 21st century skills in their department? Regarding this aspiration to find answers to the questions above, this study aims to investigate the general structure of ELT programs, and to analyze the distinctness among ELT departments in different universities. In addition to this, the perceptions and ideas of students and academics of the participating ELT programs in Turkey are interrogated. 3. Method 3.1. Research design This study utilized mixed methods based on both qualitative and quantitative data. For granting validity, a triangulation model was applied for the data collection and the analysis of the data, triangulation refers to the use of multiple methods or data sources in qualitative research to develop a comprehensive understanding of phenomena (Patton, 1999; Carter et al., 2014). With an aim to establish an extensive understanding of the study, triangulation was applied for the data collection and analysis. With this respect, the data set of the study has 3 branches: • Curricula analysis of the ELT departments’ course packs, • Quantitative data analysis of the questionnaires, • Content analysis of the interview sessions. 3.2. Study Group The participants of the study were composed of 10 academics and 67 students in the field of ELT selected via convenience sampling method from 4 different universities in Ankara, Gaziantep, İzmir and İstanbul which were selected using convenience sampling method. The convenience sampling method was used which could be defined as selection of the participants who were available and volunteer (Martinez& Martinez, 2001) and which is a way of embracing participants who are reachable to the researcher (Etikan, 2016). Saunders, Lewis, and Thornhill (2012) describe that in convenience sampling it is feasible to encourage all subjects to participate in the study without applying any specific criteria. The number of the participants were randomly distributed to the selected four universities. Therefore, 77 participants constituted the study group of the study. In the study, 2 students and 1 academic from each university participated in the interview sessions. Thus, 4 academics and 8 students in total took part in the interview sessions of the study. The student participants were at their 3rd and 4th year at the university. 79% of the student participants were females in this study. On the other hand, 40% of the academics were females in the study. In addition to this, the working experience of the academics are heterogeneous varying from 7 years to 23 years. Both student and academic participants dispersed heterogeneously to the 4 cities (Ankara, Gaziantep, İzmir, İstanbul). For the curricula review, 4 different university’s ELT departments were selected via convenience sampling method. The Ataberk & Mirici 1518 European Credit Transfer System (ECTS) information packs of the universities and course contents are available in related universities’ websites. The obtained data were analyzed using qualitative content analysis to see the content of the programs is in terms of 21st century skills development. 3.3. Data Collection Tools In the study, 3 different types of data collection tools were utilized with an aim to investigate the understanding of the perspectives of the academics and students considering the 21st century skills and their practices throughout their curricula, as well as the accordance with their course contents and objectives in ELT curricula of the selected departments. These data collection tools consisted of; • analysis of the online course contents • questionnaires (questionnaire for academics & questionnaire for students), • interview forms (interview form for academics & interview form for students), . 1. Questionnaires a. Academics’ questionnaire The academics’ questionnaire consisted of 3 sections: personal information, developing 21st century skills and an open-ended item. The Personal Information section asked the gender, institution, years of experience in the field and courses to teach to the academics in the field of ELT. Then, the academics’ questionnaire had Developing the 21st Century Skills section with 4 sub-dimensions as follows: • Statements about the curricula in your department, • Statements about life and career skills, • Statements about learning and innovation skills, • Statements about information, media, and technology skills. Finally, the academics’ questionnaire included an open-ended item which interrogated opinions about the most important 21st century skills that the students needed to develop. In the first draft of the questionnaire, it involved 47 items. After the experts’ opinion, necessary changes were made on the items based on the feedback from the expert. This way, the quality and the validity of the questionnaire was ensured. Considering the time and focus of the participants, it was suggested to eliminate some of the items so that it could be a more precise and direct questionnaire. In addition to that, the ‘Developing the 21st Century Skills’ section was proposed to be classified as the 4 sub-dimensions listed above. The questionnaire used a 5-point Likert scale for the answers, and it had 33 items and an open-ended item for participants to answer. At the beginning of the items, it was clearly stated that participants did not need to share their personal information (e.g. name-surname), the answers would be accessible only for the researcher, they could answer the questionnaire completely on a voluntary basis, which meant that they were free to not to answer or to quit at any part of the questionnaire. Ethics commission application has been approved with the necessary International Online Journal of Education and Teaching (IOJET) 2022, 9(4), 1513-1544. 1519 documents. The academics’ questionnaire aimed to highlight academics’ perceptions on the curriculum and development of 21st century skills. b. Students’ questionnaire The students’ questionnaire consisted of 3 sections: personal information, developing 21st century skills and an open-ended item. Personal Information section asked the gender, institution, and the grade to the students in the field of ELT. Then, the students’ questionnaire had Developing the 21st Century Skills section with the same 4 sub-dimensions and the 5- point Likert scale for the answers as given in the academics’ questionnaire. The academics’ questionnaire aimed to highlight academics’ perceptions on the curriculum and development of 21st century skills. The students’ questionnaire aimed to understand students’ perceptions on 21st century skills and the sub-skills in terms of the curriculum. Table 5 presents the structure of the academics and students’ questionnaires. Table 1. The structure of the academics and students’ questionnaires Academics’ Questionnaire Students’ Questionnaire S e c ti o n s 1. Personal Information 1. Personal Information 2. Developing the 21st Century Skills 2.1. Statements about the curricula in your department 2.2. Statements about life and career skills 2.3. Statements about learning and innovation skills 2.4. Statements about information, media, and technology skills 2. Developing the 21st Century Skills 2.1. Statements about the curricula in your department 2.2. Statements about life and career skills 2.3. Statements about learning and innovation skills 2.4. Statements about information, media, and technology skills 3. Open-ended item 3. Open-ended item As can be concluded from Table 1, Academics’ Questionnaire and Students’ Questionnaire are, in fact, overlapping forms. However, they aim to interrogate the opinions, beliefs and perspectives related to 21st century skills from academics and students’ perspectives respectively. 2. Interview forms In the study, two interview forms were utilized: one for the academics, the other for the students in the selected ELT departments. Interview forms have been developed by the researcher and finalized after the experts’ opinion and making the necessary revisions in the items based on the experts’ feedback in terms of wording and the statements. This way, the quality and the validity of the form was ensured. The interview forms had 4 open-ended questions. The participants were expected to share their ideas and experiences orally Ataberk & Mirici 1520 throughout the interview. At the beginning of the interview, participants were informed about the confidentiality of their answers. The researcher asked for their approval to record the meeting. After the approval, the researcher stated that all kinds of personal information would be kept anonymous, the answers would be used only for academic purposes and only by the researcher, they were completely free to join or quit the interview, they could stop the interview at any part, or they could skip the questions if they would not feel comfortable answering it. 3. Website Search In order to determine the program content about 21st century skills development, document analysis method was utilized. For the website search, online course catalogs of the universities were accessed via the websites of each department. In the online course catalogs, course titles, objectives and contents were taken into consideration for analysis. 3.4. Data Analysis The analysis of the obtained data was conducted as in the following. a. Analysis of the Questionnaires In this research, 2 different questionnaire forms were utilized for students and academics of Turkish ELT Programs. Each questionnaire was a 5-point Likert-scale type consisting of 33 items as well as an open-ended item. The main aim of the questionnaires was to determine the beliefs, opinions and attitudes of the students and academics about the English Language Teaching programs and curricula in Turkey. The questionnaires consisted of 3 sections: 1. Personal Information, 2. Developing 21st Century Skills items, 3. Open- ended item. In the second section, developing 21st Century Skills, the 5-point Likert-scale type items were categorized as: a. Statements about the curricula in the department, b. Statements about life and career skills, c. Statements about learning and innovation skills, d. Statements about information, media, and technology skills. The categories included 11 items, 8 items, 8 items, 6 items consecutively. The items were analyzed using T-test, Spearman's rank correlation coefficient, Pearson product-moment correlation coefficient, Mann Whitney U tests. T-test is mainly used for the comparison of two means while Spearman's rank correlation coefficient which is ‘essentially equivalent to the usual correlation coefficient calculated on the ranks of the two variables (say X and Y) within their respective samples.’ (CFA Institute, 2019, p.645). Pearson product-moment correlation coefficient was defined as ‘a measure of the strength of the linear relationship between two variables.’ (French& Torres Ronda, 2021, p. 268) and was ‘computed through examination of the proportion of variance that is shared by variables.’ (Miller& Salkind, 2002). Finally, Mann Whitney U tests were based on the assumptions related ‘to independency of samples and the ordinal nature of data’ (Bajpai, 2009, p.684). As a means to perform Mann Whitney U test, ‘the sample values were combined into one group and then, those values were arranged in ascending order.’ (Bajpai, 2009, p.684). The open-ended item of the questionnaire aimed to answer the most highlighted 21st century skills by the students and academics. In the questionnaires, option 1 represented “strongly disagree” and option 5 represented “strongly agree”. 67 students and 10 academics have participated in the study. International Online Journal of Education and Teaching (IOJET) 2022, 9(4), 1513-1544. 1521 For the academics’ questionnaire, 40% of the participants was female while 60% of it was male. For the students’ questionnaire, 79,1% of the participants was female while 20,9% of them constituted the male participants of the study. b. Analysis of the Interview Forms The interview forms and the documents were analyzed qualitatively by using the content analysis method after the transcription of the interviews. The transcriptions were verbatim. Content analysis was performed to establish recurring codes and themes, that makes it, emergent codes, in other words, emergent themes was utilized. As Stemler (2001) mentioned, in emergent themes, themes are assembled after the texts, with categories and their subcategories according to the recurring, frequent codes. The data of both the curricula and the interviews were grouped via their categorization and were analyzed and interpreted as the findings of the study. Interviews were conducted both face-to-face and via online platforms and the meetings were recorded by the researcher. 1 academic and 2 students from each university has participated in the interviews. The interviews took approximately 5 minutes on average and the responses to the questions in the interview forms were transcribed by the researcher for the analysis. Following the transcription, most recurring themes and vocabulary were highlighted by the researcher and the responses were grouped based on their theme and emphasis. c. Analysis of the data obtained via the website search In the study, course packs of 4 different Faculties of Education were analyzed through document analysis method. Course description catalogs of the aforementioned universities are accessible online, in their related websites. This part of the study is limited to the online course catalogs of the universities. For the analysis of this section, course objectives and course contents were taken into consideration. The recurring themes and vocabulary were highlighted in each of the universities’ online course catalogs and their correlation with the course objectives were investigated. Emergent themes and codes were utilized by the researcher. 4. Findings 4.1. Analysis of the quantitative data The main purpose of the 2 questionnaire forms is to have a better understanding in perceptions of ELT academics and students related to 21st century skills and the skills position in the English Language Teaching curricula. 67 responses from the students’ questionnaire and 10 responses from the academics’ questionnaire were analyzed by using SPSS data analysis program. Before proceeding to the analysis of the research questions on the data collected within the scope of the research, a missing value analysis was carried out. In the missing value analysis, it was seen that there was no missing value in the data set. After the missing data analysis, reverse items in the scales (Item 13, Item 16, Item 19, Item 23, Item 26, Item 29, Item 32 for Student, and Instructor Forms) were transformed. After these processes, the normality of the data was examined with the mean, median, peak value, skewness, and kurtosis coefficients of the sub-dimensions of the scales. These values are given in Table 2. Ataberk & Mirici 1522 Table 2. Normality Assumption Values for Examined Statistics �̅� Xavg Peak Value Skewness Kurtosis S tu d e n ts ’ F o r m Curricula 40.49 40.00 38.00 .14 -.56 Life and Career Skills 29.03 29.00 29.00 -.05 -.05 Learning and Innovation Skills 32.25 32.00 29.00 .35 -.76 Information, Media and Technology Skills 20.57 21.00 21.00 -.17 .42 A c a d e m ic s’ F o r m Curricula 45.60 45.50 45.00 -1.18 2.45 Life and Career Skills 29.50 30.00 30.00 -.08 -.88 Learning and Innovation Skills 31.40 32.00 29.00 -.76 .30 Information, Media and Technology Skills 19.40 19.50 21.00 -.43 -1.47 When Table 2 is examined, it is seen that the data for the academics’ form display a normal distribution, except for the Information, Media, and Technology Skills sub- dimension. For this reason, parametric methods will be used in the analyzes where the said data will be used, and non-parametric methods will be used in the remaining data (Tabachnick & Fidell, 2013; Pallant, 2016). In order to reveal the structure of the study group, the demographic information of the study group is given in Table 3. Table 3. Demographic Information Questionnaire Variant Sub-dimension n % S tu d e n ts ’ Q u e st io n n a ir e Gender of the Student Female 53 79.1 Male 14 20.9 Year of the student at university 2 2 3.0 3 30 44.8 4 35 52.2 A c a d e m ic s’ Q u e st io n n a ir e Gender of the academician Female 4 40.0 Male 6 60.0 Academician’s year of experience at university 7 1 10.0 12 2 20.0 13 2 20.0 16 1 10.0 17 1 10.0 21 2 20.0 23 1 10.0 International Online Journal of Education and Teaching (IOJET) 2022, 9(4), 1513-1544. 1523 1. Student Levels Regarding the Curriculum Sub-Dimension of the Students’ Questionnaire Form of the 21st Century Skills Form In order to find an answer to the research question expressed as “To what extent are the 21st century skills included in the curricula of English Language Teaching programs in Turkey after the Council of Higher Education announced the autonomy of universities in terms of designing their own syllabuses?” among the research questions in the “Students’ Questionnaire” answers were examined. As a result of the examination, it was seen that the average of the students regarding the mentioned sub-dimension was 40.49. Considering that the total score that can be obtained from the curriculum sub-dimension is 55, when this average is interpreted, it can be stated that the students expressed their opinion that 21st century skills are included in the curriculum. When the results of the aforementioned sub- dimension are examined from another perspective, it is seen that the median of the ‘Curriculum’ sub-dimension is 33, and it is seen that 82.1% of the participants responded above this value. 2. Student Levels Regarding the Curriculum Sub-Dimension of the Academics’ Questionnaire Form of the 21st Century Skills Scale In order to find an answer to the research question expressed as “To what extent are the 21st century skills included in the curricula of English Language Teaching programs in Turkey after the Council of Higher Education announced the autonomy of universities in terms of designing their own syllabuses?” among the research questions in the “Academics’ Questionnaire” answers were examined. As a result of the examination, it was seen that the average of the instructors regarding the mentioned sub-dimension was 45.60. Considering that the total score that can be obtained from the curriculum sub-dimension is 55, when this average is interpreted, it can be stated that the instructors expressed their opinion that 21st century skills are included in the curriculum. When the results of the aforementioned sub- dimension are examined from another perspective, it is seen that the midpoint of the ‘Curriculum’ sub-dimension is 33, and it is seen that 100% of the participants, that is, all of them, responded above this value. 3. Student Levels Regarding the Life and Career Skills Sub-Dimension of Students’ Questionnaire Form of the 21st Century Skills Scale In order to find an answer to the research question expressed as “What is the level of students' views on life and career skills within the scope of 21st Century Skills?” among the research questions in the “Students’ Questionnaire” answers were examined. As a result of the examination, it was seen that the average of the students regarding the mentioned sub- dimension was 29.03. Considering that the total score that can be obtained from the Life and Career Skills sub-dimension is 40, when this average is interpreted, it can be stated that the students express their opinion that they have life and career skills, one of the 21st century skills. When the results of the aforementioned sub-dimension are examined from another perspective, it is seen that the midpoint of the Life and Career Skills sub-dimension is 24, and it is seen that 86.6% of the participants responded above this value. Ataberk & Mirici 1524 4. Student Levels of the Life and Career Skills Sub-Dimension of the Academics’ Questionnaire Form of the 21st Century Skills Scale In order to find an answer to the research question expressed as “What is the level of the academics' views on students' life and career skills within the scope of 21st Century Skills?” among the research questions in the “Academics’ Questionnaire” answers were examined. As a result of the examination, it was seen that the average of the instructors regarding the aforementioned sub-dimension was 29.50. Considering that the total score that can be obtained from the Life and Career Skills sub-dimension is 40, when this average is interpreted, it can be stated that the academics expressed the opinion that the students have life and career skills, one of the 21st century skills. When the results of the aforementioned sub-dimension are examined from another perspective, it is seen that the mid-point of the Life and Career Skills sub-dimension is 24, and it is seen that 100% of the participants, that is, all of them, responded above this value. 5. Student Levels Regarding the Learning and Innovation Skills Sub-Dimension of the Students’ Questionnaire Form of the 21st Century Skills Scale In order to find an answer to the research question expressed as “What is the level of students' views on learning and innovation skills within the scope of 21st Century Skills?” among the research questions in the “Students’ Questionnaire” answers were examined. As a result of the examination, it was seen that the average of the students regarding the mentioned sub-dimension was 32.25. Considering that the total score that can be obtained from the Learning and Innovation Skills sub-dimension is 40, when this average is interpreted, it can be stated that the students express their opinion that they have learning and innovation skills, one of the 21st century skills. When the results of the aforementioned sub-dimension are examined from another perspective, it is seen that the mid-point of the Learning and Innovation Skills sub-dimension is 24, and it is seen that 100% of the participants, that is, all of them, responded above this value. 6. Student Levels Regarding the Learning and Innovation Skills Sub-Dimension of the Academics’ Questionnaire Form of the 21st Century Skills Scale In order to find an answer to the research question expressed as “What is the level of the academics’ views on the students' learning and innovation skills within the scope of 21st Century Skills?” among the research questions in the “Academics’ Questionnaire” answers were examined. As a result of the examination, it was seen that the average of the instructors regarding the aforementioned sub-dimension was 31.40. Considering that the total score that can be obtained from the Learning and Innovation Skills sub-dimension is 40, when this average is interpreted, it can be stated that the instructors expressed the opinion that the students have learning and innovation skills, one of the 21st century skills. When the results of the aforementioned sub-dimension are examined from another perspective, it is seen that the mid-point of the Learning and Innovation Skills sub-dimension is 24, and it is seen that 100% of the participants, that is, all of them, responded above this value. International Online Journal of Education and Teaching (IOJET) 2022, 9(4), 1513-1544. 1525 7. Student Levels Regarding the Information, Media, and Technology Skills Sub- Dimension of the Students’ Questionnaire Form of the 21st Century Skills Scale In order to find an answer to the research question expressed as “What is the level of students' views on information, media, and technology skills within the scope of 21st Century Skills?” among the research questions in the “Students’ Questionnaire” answers were examined. As a result of the examination, it was seen that the average of the students regarding the mentioned sub-dimension was 20.57. Considering that the total score that can be obtained from the Information, Media and Technology Skills sub-dimension is 30, when this average is interpreted, it can be stated that the students express their opinion that they have 21st century skills such as knowledge, media, and technology. When the results of the aforementioned sub-dimension are examined from another perspective, it is seen that the mid-point of the Learning and Innovation Skills sub-dimension is 18, and it is seen that 76.1% of the participants responded above this value. 8. Student Levels Regarding the Information, Media, and Technology Skills Sub- Dimension of the Academics’ Questionnaire Form of the 21st Century Skills Scale In order to find an answer to the research question expressed as “What is the level of the academics’ views on students' knowledge, media, and technology skills within the scope of 21st Century Skills?” among the research questions in the “Academics’ Questionnaire” answers were examined. As a result of the examination, it was seen that the average of the instructors regarding the mentioned sub-dimension was 19.40. Considering that the total score that can be obtained from the Information, Media and Technology Skills sub-dimension is 30, when this average is interpreted, it can be stated that the instructors expressed their opinion that the students have the 21st century skills of knowledge, media, and technology. When the results of the aforementioned sub-dimension are examined from another perspective, it is seen that the mid-point of the Learning and Innovation Skills sub-dimension is 18, and it is seen that 70% of the participants responded above this value. 9. The Relationship Between 21st Century Skills Scale Students’ and Academics’ Questionnaire Forms Correlation coefficients for each sub-dimension of the student and instructor forms were calculated in order to find an answer to the research question expressed as “Is there a relationship between the 21st century skills scale student and instructor forms?” among the research questions. In this context, Spearman Brown Rank Differences Correlation Coefficient was calculated in the calculations related to the curriculum sub-dimension of the academics’ form, while Pearson Product-Moment Correlation Coefficient was calculated among other sub-dimensions. As a result of the analysis, it was seen that there was no significant relationship between students’ and academics’ sub-dimensions. rrho(Curriculum)=.35, p=.33, r(Life and Career Skills)=-.02, p=.96, r(Learning and Innovation Skills )=.04, p=.92, r(Information, Media, and Technology Skills)=-.11, p=.76. Ataberk & Mirici 1526 10. Examining the Differences in the Total Scores of the 21st Century Skills Scale Curriculum Sub-Dimensions by being a Student and Academic In order to find an answer to the research question expressed as “Do the total scores of the 21st Century Skills Scale Curriculum sub-dimension differ according to the status of being a student and an academic?”, Mann Whitney U Test was conducted. Table 9 shows the statistics of the results of the Mann Whitney U test, which was conducted to determine whether the total scores of the 21st Century Skills Scale Curriculum sub-dimension differ significantly according to the status of being a student and an academic. Table 4. Mann Whitney U Results on the Differences in 21st Century Skills Scale Curriculum Sub-Dimension Total Scores According to Being a Student and Instructor Group N Row Sum Rank Average U Z p Students 67 2462.50 36.75 184.50 -2.28 .02 Academics 10 540.50 54.05 Total 77 When Table 4 is examined, it is seen that the total scores of the 21st Century Skills Scale Curriculum sub-dimension differ significantly according to the status of being a student and an academic, U=184.50, p=.02. (Pallant, 2016). When the mean rank is examined, it is seen that the instructors ((X=) ̅54.05) have a higher curriculum sub-dimension total score than the students (X ̅=36.75). In other words, it can be stated that the views of the instructors regarding the inclusion of 21st century skills in the curriculum are at a higher level than the students. 11. Examination of the Differences in the Total Scores of the Life and Career Skills Sub-dimension of the 21st Century Skills Scale according to the Status of Being a Student and an Academic T-Test for Independent Groups was conducted in order to find an answer to the research question expressed as “Do the total scores of the Life and Career Skills sub- dimension of the 21st Century Skills Scale differ according to the status of being a student and an instructor?”. Table 10 shows the statistics of the results of the T-test for Independent Groups conducted to determine whether the total scores of the Life and Career Skills sub- dimension of the 21st Century Skills Scale differ significantly according to the status of being a student and an academic. Table 5. T-Test Results for Independent Groups Regarding the Differences in 21st Century Skills Scale Life and Career Skills Sub-Dimension Total Scores According to Being a Student and Instructor Group N �̅� SS sd t p Students 67 29.03 4.41 75 -.32 .75 Academics 10 29.50 3.66 International Online Journal of Education and Teaching (IOJET) 2022, 9(4), 1513-1544. 1527 When Table 5 is examined, it is seen that the total scores of the 21st Century Skills Scale Life and Career Skills sub-dimension do not differ significantly according to the status of being a student and an academic, t(75)=-.32, p=.75 (Pallant, 2016). 12. Examining the Differences in 21st Century Skills Scale Learning and Innovation Skills Sub-Dimension Total Scores by Being a Student and an Academic T-Test for Independent Groups was conducted in order to find an answer to the research question expressed as “Do the total scores of the 21st Century Skills Scale Learning and Innovation Skills sub-dimension differ according to the status of being a student and an instructor?”. Table 11 shows the statistics of the results of the T-test for Independent Groups, which was conducted to determine whether the total scores of the 21st Century Skills Scale Learning and Innovation Skills sub-dimension differ significantly according to the status of being a student and an academic. Table 6. T-Test Results for Independent Groups Regarding the Differences in 21st Century Skills Scale Learning and Innovation Skills Sub-Dimension Total Scores According to Being a Student and an Academic Group N �̅� SS sd t p Students 67 32.25 3.50 75 .72 .47 Academics 10 31.40 3.66 When Table 6 is examined, it is seen that the total scores of the 21st Century Skills Scale Learning and Innovation Skills sub-dimension do not differ significantly according to being a student and an instructor, t(75)=-.72, p=.47 (Pallant, 2016). 13. Examining the Differences in 21st Century Skills Scale Information, Media and Technology Skills Sub-Dimension Total Scores by Being a Student and an Academic T-Test for Independent Groups was conducted in order to find an answer to the research question expressed as “Do the total scores of the 21st Century Skills Scale Information, Media and Technology Skills sub-dimension differ according to the status of being a student and an instructor?”. Table 12 shows the statistics of the results of the T-test for Independent Groups, which was conducted to determine whether the total scores of the Information, Media, and Technology Skills sub-dimension of the 21st Century Skills Scale differ significantly according to the status of being a student and an academic. Table 7. T-Test Results for Independent Groups Regarding the Differences in 21st Century Skills Scale Information, Media and Technology Skills Sub-Dimension Total Scores According to Being a Student and an Academic Group N �̅� SS sd t p Students 67 20.57 2.96 75 1.22 .23 Academics 10 19.40 1.65 Ataberk & Mirici 1528 When Table 7 is examined, it is seen that the total scores of the 21st Century Skills Scale Information, Media and Technology Skills sub-dimension do not differ significantly according to the status of being a student and an academic, t(75)=1.22, p=.23 (Pallant, 2016). Analysis of the qualitative data Academics’ and students’ interview forms constituted the first part of the qualitative data in this study. The second part of the qualitative data was the website search aiming to analyze the curricula of ELT programs via document analysis method. In this part, findings of the interview forms and the website search will be provided. First interview question First interview question in the form was “Which classes and activities do you have about the development of 21st century skills?”. According to the responses of the participants, the academics engaged some of their classes with 21st century skills. All of the participants have stated that they have purposeful classes and activities for the development of the skills. Mostly, they used the words “utilize”, “essential”, “beneficial” and “helpful”. “Among the classes I teach, I find it helpful to include activities about the development of 21st century skills.” (P2) “I have different kinds of activities that aim to enhance the prospective teachers’ command in 21st century skills and I believe this is essential for teacher training. We mainly utilize group-work tasks for the assessment of the courses, and I encourage my students to conduct research before the class in technological setting so that they can filter the information mess in the online world.” (P3) The responses that were given to the first interview question of the interview form has shown that the academics tend to include 21st century skills and they perceive the skills as beneficial and helpful. These skills are essential for a 21st century student and for a 21st century teacher so they correlate their classes with the skills in order to create a helpful teaching and learning for the practice of 21st century skills. Second interview question The second interview question of the form was “Do you adapt your course content to include any kind of 21st century skills?”. All of the participants responded to this item positively. They stated they would make necessary arrangements for their course content. “I always read new resources and keep up with the publications. I get inspired by the innovative studies and at the beginning of each semester, I re-design my course syllabuses with an aim to enrich my course materials.” (P1) “Definitely. In my opinion, course contents should be living, and they should be open for updates. That is how I perceive my classes.” (P4) The two excerpts above represent that the academics pay special attention to keeping the course content up-to-date and enriching to meet the contemporary needs of the era such as the 21st century skills. International Online Journal of Education and Teaching (IOJET) 2022, 9(4), 1513-1544. 1529 Third interview question The item “Do you think your program is adequate for developing 21st century skills? If not, what can be done to improve this?” constituted the third question of the interview. According to the answers to the question number 3, the participants mainly thought the program is open to improvements in terms of 21st century skills although it has some instances. They used the words “enrich”, “upgraded”, and “developed”. Development of training for 4C skills is also underlined. “Our program has beneficial aspects for training of contemporary teachers. However, it should be developed. Individual endeavor to enhance the course contents is definitely useful but not adequate. English Language Teacher Training programs should have a holistic curriculum that is equipped with the 21st century skills. From the very first semester, the students should be exposed to direct training of commanding the skills.” (P1) “I cannot say that this program does not help at any part of developing the 21st century skills. However, I cannot say it is adequate or satisfactory, either. It gets better closer to graduation but in the first years of the students’ teacher training programs, they signal us that they need some support to practice 21st century skills. Specifically, they need more input in terms of developing 4Cs.” (P2) “By looking at the curriculum, the courses have strong points for development of 21st century skills. I try to contribute to this in and out of the class by updating my course contents and I know many colleagues who tries to do the same. Interestingly, when I observe the students, I interpret that this is not adequate. For instance, they conduct groupwork tasks so that they can practice their communication and collaboration skills but when they had a problem in the group, they suffer in compromising, finding solutions and solving problems. I think this is a valuable indicator.” (P3) “It is not adequate, and it is open for developments. We try to perform our best so that we can be useful for our students’ future professional lives, but it can be developed. Training of 4C’s could be a matter of regulation.” (P4) It can be concluded from the excerpts above that the academics think their curricula has enhancing aspects considering the 21st century skills. However, they make crucial points with an aim to make it more comprehensive and holistically including the 21st century skills. Fourth interview question Last item of the interview form was “Do you want to share any further insights?” and it purposed to reveal the academics’ further opinions that were not touched upon directly with the interview questions. 2 of the participants responded to the question and they highlighted the importance of aforementioned statements. “We need to detect if our students have sufficient amount of information about the structures of the skills. Maybe, we can start with introducing them with 21st century skills before asking them to practice by living. This could raise awareness as they get accustomed to 21st century skills.” (P2) “I find these studies crucial and critical in terms of taking innovative steps. What we do here in English language teacher training programs is precious since we perform our bests to train contemporary, open-minded, and knowledgeable English language teachers. Our students have the basis for being 21st century teachers and they have a chance to Ataberk & Mirici 1530 practice it with tasks. I believe they are quite equipped with the skills but their readiness levels for new learnings are amazing. Therefore, if there is anything that we can do more to inspire the students, we should go after it.” (P4) The two excerpts above demonstrate that the practice and presence of the 21st century skills are a critical and an essential point of teacher training. Students’ interview forms Students’ interview form included four open-ended questions similar to the academics’ form. The interviews were conducted with 8 students from 4 different universities. The first part of the qualitative data of the study was collected with both face-to- face and online meetings. The open-ended questions in the interview form aimed to highlight the students’ perceptions and beliefs about development of 21st century skills in their curricula and to what extent do they correlate their courses with 21st century skills. Students’ participation in the study and their points of views to the interview questions have contributed positively to the qualitative data of the study. The purpose of the interview form is to be able to have a comprehensive examination about the perceptions of the 21st century skills and their teaching programs among the 8 participants. First interview question Students of ELT programs’ first interview question was “Which classes and activities do you have about the development of 21st century skills?” The participants listed various courses associated with 21st century skills. They responded positively to the question as they explained their ideas and statements about the courses and 21st century skills. The participants listed the courses that are (a) directly about communication, collaboration, and technology, (b) related to social contexts and society as they can practice the skills as a vehicle to participate in the class and, (c) designed for students’ applied micro-teaching courses. Participants mentioned that they utilize 21st century skills when they prepare their lesson plans, course contents, and materials for presentation. “We have Materials Developments class which is about using technological vehicles to create tasks and use them in classrooms and adapt and integrate technology in our classes.” (P1) “We had Oral Communication Skills as well as critical thinking lessons. Also, I had elective courses for 4 skills of English learning and teaching. I think I can count 5 lessons among the ones I have taken.” (P4) “I had classes about Teaching English to Young Learners, Teaching Literature and Language, the classes about English skills and Material Design. These are 4 of the courses I have taken this semester and I think they are related to 21st century skills which have social skills.” (P6) “We had Oral Communication Skills, Critical Reading and Writing, Drama in English Language Teaching, Materials Design for English Language Teaching, World Englishes and Culture classes and I think they can be related to 21st century skills since we accomplished many different tasks for them like groupwork presentations and lesson plans, designing and conducting online and technology-based tasks.” (P7) International Online Journal of Education and Teaching (IOJET) 2022, 9(4), 1513-1544. 1531 “I think I can list Critical Reading and Writing, English in Mass Communication, Language and Society, Teaching English to Young Learners, Course Content Development in English Language Teaching. Applied courses are much more encouraging because we both have a chance to get used to 21st century skills and we practice how can we utilize them as English teachers during micro-teachings.” (P8) Judging from the excerpts above, the students’ classification of the courses specified their opinions related the presence of the skills in their courses. They deliberately listed the course names as they believe that they can practice the skills. Second interview question The second question of the interview form was “Do your professors adapt class content to include 21st century skills activities?” Participants stated that this is not valid for all professors and classes. According to the responses of the participants, some of the courses are still open to improvement while some of the courses are beneficial and prosperous in terms of communication, collaboration, and ICT skills. 2 of the participants have related this question to Covid-19 pandemic. While one of the participants stated that the pandemic helped development of certain skills, the other participant stated that because of the pandemic, certain skills were weakened, and it needs more training now to regain the skills. “Considering the pandemic, most of our teachers used technology in our classes and we used applications like Zoom or Microsoft Teams to improve ourselves in classrooms like Language Acquisition. That is why I think, nowadays most of our classes are technology- based.” (P2) “I think it depends on the lesson and on the professor. But I have had lessons in which my professors adapt the lessons for 21st century skills but I also had lessons that they did not. So, it is like 50 per-cent.” (P3) “Yes. In all the lessons I just listed, we work in groups, and it helps our social skills since the pandemic broke out in 2020, the response and reaction speed has been affected negatively for all of us. So, I think, being in groups helps us improve our skills.” (P6) “I cannot say all of our professors do this. We have quite interactive classes about the 21st century skills but we also still have very straight-forward, lecture-type classes. Some of our professors do not adapt the classes, I think. Courses like Listening Skills, Reading Skills, Linguistics, English Literature are not that interactive, and I think they are weak in terms of 21st century skills.” (P8) According to the students, some of the courses are still open to advancement while some of the courses are helpful and constructive in terms of the 21st century skills. On the other hand, two of the participants highlighted the effects of Covid-19 pandemic from different aspects. Third interview question Thirdly, participants responded to the question “Do you feel qualified about 21st century skills? If not, what can help you achieve better command of 21st century skills?”. Most of the participants remained abstained while responding to this question. 7 of the participants stated that although they feel good at some points, they have more to learn and practice while only 1 of the participants stated that he was feeling qualified enough to Ataberk & Mirici 1532 command 21st century skills. The participants mostly tend to answer this question in terms of technological aspects, but they also took social skills into consideration as well. Participants stated and indicated that more technology-focused trainings would be useful for them. “I cannot answer this question right now because 21st century skills I mean technology, is a system that gradually improves itself and you need to adapt yourself to the technology. To do this, you need to study, you always need to read. So, I think I am not capable of using it as I needed to do so. But I think I can conduct some research and do my assignments. I am kind of capable of doing that.” (P1) “From time to times, yes. I feel that I am qualified but mostly I can say that I feel qualified, but the thing is that I am not very good with technology. Maybe that can be a little tricky for me. Except for the technological skills, I can say that I feel qualified yes.” (P4) “I feel qualified, and I am confident about it. I think I am good at social skills; I can always think outside the box, and I am good at technology. So, I think I am good.” (P5) “I do not feel that I am qualified about 21st century skills in terms of technology and social skills, both. I think I can be more improved and educated better.” (P6) According to the responses of the participants, 87.5% of the participant state that they have a lot to proceed as to practice the 21st century skills. They state that the courses include the skills, and the professors attempt to encompass the skills in their course contents. However, seven of the participants do not feel completely qualified concerning the 21st century skills while one of the participants state that he has a good command of the skills. Fourth interview question Last item of the interview form was “Do you want to add anything else?” and it purposed to reveal the students’ further opinions, ideas, and comments that were not touched upon directly with the interview questions. 3 of the participants preferred to respond to this question and they stated their suggestions for ELT programs in Turkey as being students of them. “I have one suggestion for this, even though we are using technology in our classes I think most of our teachers are not capable of using applications like Zoom or Microsoft Teams. So, we need to start teaching them in our classes so that we can use them when we grow up and become teachers.” (P1) “Our course list should be more diverse than that. We can have many elective courses focusing on the 21st century skills but we are stuck with the same ones every year. Training of this can be an important part of training as teachers.” (P5) “I was in International Bachelorette (IB) program in high school. I realized that compared to the Turkish education system, 21st century skills go better with IB because of the whole different approaches but because of Turkish education is still based on memorizing. I think IB has more option to adapt contents that is why I think it goes better.” (P7) The excerpts above represent the students’ perceptions about the ELT programs and education system in Turkey. Website Search Online ECTS Course Information Packs of 4 universities were analyzed as the second part of qualitative data of the study. In the websites of the related universities, course International Online Journal of Education and Teaching (IOJET) 2022, 9(4), 1513-1544. 1533 contents and objectives are listed and defined. Among those definitions, certain words are highlighted with an aim to understand the course’s contributions to development of 21st century skills. This part of the study is limited to the online resources for ECTS Course Information Packs. As a general framework, when the course packs of the aforementioned 4 universities are examined, it was seen that the universities have similar courses for similar semesters. In addition to this, the course lists are in concordance with Council of Higher Education’s new English Language Teaching Curricula. The courses which are the same among the universities have similar contents according to the ECTS Information Pack of the related university. Table 8 shows the course lists of the 4 universities related to reading and writing skills in the first semester of their ELT programs. Table 8. Departmental Compulsory Courses Related to Reading and Writing Skills in the First Semesters of ELT Programs Name of the university ECTS- TPC Course name University 1 (U1) 2- 202 Reading Skills I University (U2) 4- 303 Advanced Reading and Writing I University 3 (U3) 2- 202 Reading Skills I University 4 (U4) 2- 202 Reading Skills I University 1 (U1) 2- 202 Writing Skills I University 3 (U3) 2- 202 Writing Skills I University (U4) 2- 202 Writing Skills I According to Table 8, the ELT programs have the same courses related to reading and writing skills of English. The only difference that can be detected for this part, U2 offers an integrated course for reading and writing skills while the other 3 universities offer different courses for the skills. When the course contents of the aforementioned courses are analyzed, it was seen that they had similar syllabus structures in general for the semester but had minor differences for course objectives. U1- Reading Skills I- “Analyzes the selected sentences by scanning the original texts on the Internet or in foreign newspapers and magazines.” U2- Advanced Reading and Writing I- “Critical thinking skills such as synthesizing information or analyzing a problem as well as reacting on the basis of evaluation are fostered.” U3- Reading Skills- “Examining original and intellectual reading passages using reading techniques” U4- Reading Skills- “Strengthening students' reading skills and encourage them to respond critically to ideas.” As can be seen from the quoted course objectives, the course objectives include the words such as “analyzes”, “critical thinking skills”, “synthesizing information”, “examining” and “respond critically to ideas” indicate us that these courses aim at training the students in terms of 21st century skills, namely for critical thinking, information literacy and media literacy skills. Table 9 shows the course lists of the 4 universities related to listening and speaking skills in the first semester of their ELT programs. This table shows departmental compulsory Ataberk & Mirici 1534 courses, and it does not include educational sciences and elective general culture courses for the related semester. Table 9. Departmental Compulsory Courses Related to Listening and Speaking Skills in the First Semesters of ELT Programs Name of the university ECTS- TPC Course name U1 2- 202 Listening and Pronunciation Skills I U2 4- 303 Listening and Pronunciation Skills I U3 2- 202 Listening and Pronunciation Skills I U4 2- 202 Listening and Pronunciation Skills I U1 2- 202 Oral Communication Skills I U2 4- 303 Oral Communication Skills I U3 2- 202 Oral Communication Skills I U4 2- 202 Oral Communication Skills I Table 9 reveals that all of the universities have the same course for listening and speaking skills with different course codes. When the course contents are examined for Oral Communication Skills I, it was understood that the courses have the same course objectives with minor variations. U1- EFL107 Oral Communication Skills I - “To develop the skills to communicate more fluently, clearly, and naturally, using appropriate expressions and strategies in various English oral communication situations.” U2- GEIN107 Oral Communication Skills I - “Communication-oriented speaking such as discussions, individual presentations and other interactive tasks “ U3- İDE107 Oral Communication Skills I - “A variety of communication-oriented speaking, such as discussion, individual presentations, and interactive activities.” U4- ELT1004 Oral Communication Skills I –“Students participate in pair and groupwork activities that will improve and trigger English speaking among their classmates.” As can be seen from the quoted course objectives, the course objectives include the words such as “communicate”, “communication-oriented speaking”, “pair and group work activities” indicate us that these courses aim at training the students in terms of 21st century skills, namely for 4C’s; communication, collaboration, critical thinking and creativity. On the other hand, the ELT curricula have distinctive courses, as well. Table 15 shows the varying departmental compulsory courses of the 4 universities. Table 10. The Varying Departmental Compulsory Courses of the 4 Universities Name of the University Course Name U1, U3, U4 Structure of English U2 Lexical Competence U1, U3, U4 Critical Reading and Writing U2 Oral Expression and Public Speaking U2 ELT Methodology I U2 ELT Methodology II U2 Drama International Online Journal of Education and Teaching (IOJET) 2022, 9(4), 1513-1544. 1535 Table 10 demonstrates that, when 8-semester ELT curricula is taken into consideration, all of the universities have the same course designs, and this is in accordance with that of Council of Higher Education’s. U2 separates itself among the other 3 universities with 5 different departmental compulsory courses. Based on the quantitative data analysis, it was found out that regarding the ‘Curriculum’ sub-dimension of the students’ questionnaire, mid-point of the ‘Curriculum’ sub-dimension is 33, and it is seen that 82.1% of the participants responded above this value with the average score of 40.49. In relation with this, when the academics’ questionnaire was analyzed in terms of ‘Curriculum’ sub-dimension, 100% of the participants responded above the mid-point, which was 33, with the average score of 45.60. Considering the ‘Life and Career Skills’ sub-dimension of the students’ questionnaire, 86.8% of the participants scored above the mid-point, which was 24, with 29.03. This could be interpreted as the students expressed that they had life and career skills, one of the 21st century skills. In relation with this, when the ‘Life and Career Skills’ sub-dimension of the academics’ questionnaire was analyzed, 100% of the participants responded above the mid-point, which was 24, with the average score of 29.50. Regarding the ‘Learning and Innovation Skills’ sun-dimension of the students’ questionnaire, 100% of the participants responded above the mid-point, which was 24, with the average score of 32.25. In connection with this, analyzing the ‘Learning and Innovation Skills’ sub-dimension of the academics’ questionnaire, similarly, 100% of the participants responded above the mid-point, which was 24, with the average score of 31.40 which could be interpreted as the students employed learning and innovation skills as one of the 21st century skills. ‘Information, Media, and Technology Skills’ sub-dimension of the students’ questionnaire was analyzed and was found out that 76.1% of the participants responded above the mid-point, which was 18, with the score of 20.57. When the academics’ questionnaire for the ‘Information, Media, and Technology Skills’ sub-dimension was analyzed, 70% of the participants responded above the mid-point, which was 18, with the score of 19.40. This could be interpreted as ‘Information, Media, and Technology Skills’ sub- dimension was the most insecure section for the students and in terms of student levels since they have the lowest averages in the data analysis. Table 16 summarizes the mid-points and the average scores of sub-dimensions in the questionnaire from students’ and academics’ perspective. Table 11. The summary of scores in students’ and academics’ questionnaire Students’ questionnaire Academics’ questionnaire Mid- Point Score Percentage Mid- point Score Percentage Curriculum 33 40.49 82.1% 33 45.60 100% Life and Career Skills 24 29.03 86.8% 24 29.50 100% Learning and Innovation Skills 24 32.25 100% 24 31.40 100% Information, Media, and Technology Skills 18 20.57 76.1% 18 19.40 70% Ataberk & Mirici 1536 Based on the qualitative data analysis, the findings were coherent with the quantitative data analysis. As the first part of the qualitative data analysis, interview forms were analyzed with content analysis method. According to the academics’ interview form, it could be interpreted as they were prone to include 21st century skills in their course contents and perceived the skills beneficial and convenient. For the current curricula, academics stated that although the curricula had enriching aspects, it could be developed or upgraded in terms of the practice of 21st century skills. When the students’ interview forms were analyzed, it could be concluded that the students perceived certain courses as equipped with the 21st century skills such as Materials Development, Oral Communication Skills, Teaching English to Young Learners, Teaching Literature and Language, Critical Reading and Writing, English in Mass Communication, Language and Society, Course Content Development in English Language Teaching. In addition to this, they did not agree that all of their professors attempted to adapt their courses accordingly, however they could spot specific courses as aforementioned. Lastly, they stated that they found themselves partially qualified regarding the skills although they could benefit from more training and practice of them. Lastly, for the website search, online ECTS course catalogs of the universities were examined via document analysis method. It was observed that the universities had similar courses for similar semesters. In addition to this, the course lists were coherent with Council of Higher Education’s new English Language Teaching Curricula (CoHE, 2018). Similar course contents and objectives were detected with different names such as ‘Advanced Reading and Writing I and Reading Skills I, Writing Skills I. When the course objectives were investigated, keywords for the practice of 21st century skills were spotted including communicating, pair and group work activities, analyzing, examining, critical thinking, synthesizing information. 3. Discussion, Conclusions and Suggestions The main research question of the study was set to analyze the general structure of ELT programs in 4 different universities in Turkey in terms of 21st century skills. According to P21 (2019a) and Ananiadou and Claro (2009); 21st century skills have 3 sub-categories which are (1) life and career skills, (2) learning and innovation skills and (3) information, media, and technology skills. Life and career skills include (a) flexibility& adaptability, (b) initiative& self-direction, (c) social& cross-cultural skills, (d) productivity& accountability, (e) leadership& responsibility. Learning and innovation skills consist of 4C’s: (a) communication, (b) collaboration, (c) critical thinking, (d) creativity. Lastly, information, media, and technology skills have 3 branches: (a) information literacy, (b) media literacy, (c) ICT (information, communications, and technology) literacy. According to Yıldız and Palak (2016), 21st century skills refer to the core competencies ‘such as reading, writing, arithmetic as well as digital literacy, critical thinking, communication, collaboration, global competencies, and problem-solving skills for students to thrive in the 21st century’ (p.77). From a different perspective, as Ledward and Hirata (2011) declare that a blend of content knowledge, specific skills, expertise, literacies constitute 21st century skills (Fandiño, 2013). In the light of these definitions, the structure of ELT curricula in Turkey can be discussed considering the 21st century skills. When the curricula analysis is taken into consideration regarding the 21st century skills, it can be International Online Journal of Education and Teaching (IOJET) 2022, 9(4), 1513-1544. 1537 inferred that some of the courses encourage the students and the professors to practice some of the skills. When the course contents are analyzed, it was seen that learning and innovation skills (4C’s) are the most appreciated part as the English language teaching and learning classrooms are open to communication, collaboration, critical thinking, and creativity by their nature. Although some of the elective general culture and educational sciences courses offer contents related to life and career skills and information, media, and technology skills; departmental compulsory courses of ELT seem to be inadequate in terms of equipping with the 21st century skills. As Ekici and Yılmaz (2013) suggests for the FATIH Project, reconstruction of the education system in the light of opportunity equality in terms of technological opportunities might be a generalized principle regarding the ELT curricula, although in the study of Shernoff et al. (2003) it is found that 81% of the school time was spent doing receptive activities such as listening to lectures, watching videos. Hence, the presence of the traditional teaching approaches is still continuing (Cox & Montgomery, 2019). The role of the curricula is essential regarding the practice of the 21st century skills and the importance of the training 21st century teachers as Darling- Hammond (2006) states that curricula must help prospective teachers develop ‘the disposition to continue to seek answers to difficult problems of teaching and learning’. The analysis of the curricula shows that each semester offers certain departmental courses enabling the academics and the students to utilize the 21st century skills. While in the first 2 semesters of the ELT programs, courses such as Reading Skills, Oral Communications Skills employs some of the 21st century skills; in the third semester, Critical Reading and Writing course employs certain skills. In the third year of ELT program, some courses including Teaching English to Young Learners, Literature and Language Teaching, Teaching of Language Skills promotes some of the 21st century skills. In addition to this, in the 4th year of ELT programs; applied courses such as Teaching Practicum, Community Service Practices reinforce the students to improve themselves with reference to the 21st century skills. Taking the overall ELT curricula into account; it can be inferred that at any grades of ELT departments in Turkey have some certain courses aiming to develop the 21st century skills. At this point, one aspect can be clarified; the courses which aim to develop the 21st century skills do not aim at completely different skills. Different courses employ similar skills which can be disputable since the student teachers of English have a chance to master at the same skills throughout the 8-semestered ELT program and this may lead to strengthening in constricted skills. If the courses of ELT curricula in Turkey integrate the skills holistically; it could be more constructive and beneficial for student teachers’ personality development as English language teachers. As a result of the data analysis, it has been revealed that the current curricula of ELT departments contain courses aiming o develop the 21st century skills at some points. However, the course contents of the current curricula are limited to some of the skills which may break the integrity of the 21st century skills in professional life reflections. The analysis results of the quantitative data and the interview forms are compatible with this document analysis method. According to the qualitative and quantitative data of the academics and students; participants consider that they have some of the components for the 21st century skills in their course contents but not completely. Students of ELT programs in Turkey assume that they have the capability of mastering at the 21st century skills in theoretical Ataberk & Mirici 1538 framework however, they do not feel utterly confident about handling the skills in a real-life application, namely because of this disruption of integrity. During the interviews, academics have stated that they were upgrading their course contents to keep it up to date and accordingly, students of ELT departments have stated that some of their course contents were being adapted by the professors. The study has some suggestions for this field in connection with the document analysis of the curricula in ELT programs in Turkey. The gaps in the course contents which cause the disruption of the integrity concerning the 21st century skills might be detected, and this might be investigated for a future curricula design for ELT programs with an aim to fulfilling the gaps. In addition to this, one category of the 21st century skills might be appointed, and the curricula analysis might be conducted in the matter of the related 21st century skills category. After the deep analysis, the curricula of ELT programs in Turkey might be compared and correlated with different ELT programs across Europe, what’s more, across the world. Lastly, as this study is designed as a mixed-methods case study in Turkey, the same methodology might be utilized for a different teacher training context in order to gain insights related to the selected country or region. Thanks to the curricula analysis section of the study, the sub- sections consisting of a curriculum might be investigated in a different teaching and learning context and this might be examined and compared with the effect of culture. International Online Journal of Education and Teaching (IOJET) 2022, 9(4), 1513-1544. 1539 References American Council on the Teaching of Foreign Languages (2012) American Council on the Teaching of Foreign Languages Proficiency Guidelines (ACTFL). Retrieved March 17, 2022 from chrome- extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.actfl.org/sites/default/file s/guidelines/ACTFLProficiencyGuidelines2012.pdf. Ananiadou, K., & Claro, M. (2009). 21st Century Skills and Competences for New Millennium Learners in OECD Countries. OECD Education Working Papers, No. 41. Paris: OECD Publishing. Akkoyunlu, B.& Baskan, G. (2015) School Principals’ Opinions on the FATIH Project in Turkey. Procedia Social and Behavioral Sciences, 174, 1497-1502. ATC21S (2012), 21st Century Skills, Retrieved April 20, 2021 from http://www.atc21s.org/ Bajpai, N. (2009). Business Statistics. Pearson. Batelle for Kids (2017) Partnership for 21st Century Skills, Retrieved April 17, 2021 from https://www.battelleforkids.org/ Bedir, H. (2019) Pre-service ELT teachers’ beliefs and perceptions on 21st century learning and innovation skills (4Cs). Journal of Language and Linguistic Studies, 15(1), 231- 246. Bhatnagar, N. (2011) Effective Communication and Soft Skills. Pearson Education India. Binkley, M., Erstad, O., Herman, J., Raizen, S., Ripley, M., & Rumble, M. (2010). Draft White Paper 1: Defining 21st century skills, p.13. Binkley,M., Erstad O., Herman J., Raizen S., Ripley M., Miller-Ricci M., & Rumble, M. (2012) Assessment and teaching of 21st century skills , Springer, Netherlands. Bradbury, A. (2012) Education policy and the ‘ideal learner’: producing recognizable learner- subjects through early years assessment. British Journal of Sociology of Education, 34(1), 1-19. Breen, M., & Candlin, C. (1980). The essentials of communicative curriculum in language teaching. Applied Linguistics, 1, 89–112. Brown, P., Lauder, H., & Ashton, D. (2008). Education, globalization, and the knowledge society. London: The teaching and learning research programme. Brumfit, C. J. (1984). Communicative methodology in language teaching: The Roles of Fluency and Accuracy. Cambridge: Cambridge University Press. Burnard P. (1991) A method of analysing interview transcripts in qualitative research. Nurse Education Today 11, 461–466. Canale, M. (1983). From communicative competence to communicative language pedagogy. In J. C. Richards & R. W. Schmidt (Eds.), Language and communication (pp. 2–27). London: Longman. Caplan, S. E. (2002). Problematic Internet use and psychosocial well-being: development of a theory-based cognitive–behavioral measurement instrument. Computers in Human Behavior, 18, 553-575. Carter, N., Bryant-Lukosius, D., DiCenso, A., Blythe, J.& Neville, A.J. (2014). The Use of Triangulation in Qualitative Research. Oncology Nursing Forum, 41(5), 545–547. CFA Institute (2019). CFA Program Curriculum 2020 Level I Volumes 1-6 Box Set. Wiley. Corporate Finance Institute (CFI)(2022), Language Proficiency Levels, Retrievet May 21, 2022 from https://corporatefinanceinstitute.com/resources/careers/resume/language- proficiency-levels/ Choy, C.S.& Cheah, P.K. (2009). Teacher perceptions of critical thinking among students and its influence on higher education. International Journal of Teaching and Learning in Higher Education, 20(2), 198-206. http://www.atc21s.org/ Ataberk & Mirici 1540 CoE. (2001). Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge: CUP Council of Higher Education (1998) Faculty of Education Teacher Training Undergraduated Programs. Retrieved May 23, 2022 from chrome- extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.yok.gov.tr/Documents/Y ayinlar/Yayinlarimiz/egitim-fakultesi-ogretmen-yetistirme-lisans-programlari-mart- 1998.pdf Council of Higher Education (2018) Undergraduate Program of English Language Teaching. Retrieved April 28, 2021 from chrome- extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.yok.gov.tr/Documents/K urumsal/egitim_ogretim_dairesi/Yeni-Ogretmen-Yetistirme-Lisans- Programlari/Ingilizce_Ogretmenligi_Lisans_Programi.pdf Cowan, S. M. (2014). Information literacy: The battle we won that we lost? portal: Libraries and the Academy, 14(1), 23–32. Cowan, S., & Eva, N. (2016). Changing our aim: Infiltrating faculty with information literacy. Communications in Information Literacy, 10(2), 163–177. Cunningham, D. (1992). Beyond educational psychology: Steps toward an educational semiotic. Educational Psychology Review. 4, 165-194. Darling-Hammond, L. (2006) Constructing 21st century teacher education. Journal of Teacher Education, 57(3), 300-314. doi: 10.1177/0022487105285962 Dent, J., Harden R.& Hunt, D. (2021) A Practical Guide for Medical Teachers, 6th edition. Elsevier, eBook ISBN: 9780702081712. Dewey, J. (1909). Moral principles in education. Boston: Houghton Mifflin. Dörnyei, Z. (2007b). Research methods in applied linguistics: Quantitative, qualitative, and mixed methodologies. Oxford: Oxford University Press. Dörnyei, Z. (2009). Questionnaires in Second Language Research:Construction, Administration, and Processing Dudley-Evans, T. & St. John M-J. (1998). Developments in English for Specific Purposes, Retrieved from: http://eltnotebook.blogspot.com/2012/01/an-eltglossary-needs- analysis.html Duron, R., Limbach, B., & Waugh, W. (2006). Critical thinking framework for any discipline. International Journal of Teaching and Learning in Higher Education, 17(2), 160-166 Ekici, S.& Yılmaz, B. (2013) An Evaluation on FATIH Project. Turkish Librarianship Journal, 27(2), 317-339. Elder, L. (2007). Why critical thinking? Retrieved, from http://www.criticalthinking.org/page.cfm?PageID=796&CategoryID=103 Elo, S., & Kyngäs, H. (2008). The qualitative content analysis process. Journal of Advanced Nursing, 62(1), 107–115. doi:10.1111/j.1365-2648.2007.04569.x Ertürk, S. (1997) Eğitimde Program Geliştirme. Edge Akademi Publishing, Ankara. Etikan, I. (2016). Comparison of Convenience Sampling and Purposive Sampling. American Journal of Theoretical and Applied Statistics. European Commission (2012a). Supporting the Teaching Professions for Better Learning Outcomes. Strasbourg, 20.11.2012. SWD (2012) 374 final. European Commission (2013a). Supporting teacher competence development for better learning outcomes. Brussels: EC. European Commission (2013b). Supporting Teacher Educators for better learning outcomes. Brussels: EC. Fandiño, Y.J. (2013). 21st century skills and the English foreign language classroom: A call for more awareness in Colombia http://www.criticalthinking.org/page.cfm?PageID=796&CategoryID=103 International Online Journal of Education and Teaching (IOJET) 2022, 9(4), 1513-1544. 1541 Flick, Uwe; Kardoff, Ernst von; Steinke, Ines. (2004). A Companion to Qualitative Research. Thousand Oaks, CA: SAGE Publications. French, D.& Torres Ronda, L. (2021). NSCA's Essentials of Sport Science. NSCA -National Strength & Conditioning Association. Garcia, C., J. (2012). Professional Development in the 21st Century – Nine Structures for Coaching and Mentoring. IDRA Newsletter. Retrieved from:https://www.idra.org/resource-center/professional-development-in-the21st- century-2/ Gaziantep University ECTS Course Information Pack (2018), Retrieved April 22, 2022 from https://www.gantep.edu.tr/en/ab/dersler.php?bolum=600&bolum_id=604 Gehrig, J. S. (2010). Patient assessment tutorials: A step-by-step guide for the dental hygienist. Philadelphia: Wolters Kluwer Health/Lippincott Williams & Wilkins. Gencer, S.L.& Koc, M. (2012). Internet abuse among teenagers and its relations to internet usage patterns and demographics. Journal of Educational Technology & Society, 15, 25-36. Goulet, L., Krentz, C.& Christiansen, H. (2003). Collaboration in Education: The Phenomenon and Process of Working Together. The Alberta Journal of Educational Research, 49(4), 325-340. Hacettepe University ECTS Course Information Pack (2018), Retrieved April 22, 2022 from https://bilsis.hacettepe.edu.tr/oibs/bologna/index.aspx?lang=tr&curOp=showPac&cur Unit=440&curSunit=785# Hammons, J. (2020). Teaching the teachers to teach information literacy: A literature review. The Journal of Academic Librarianship, 46, 1-13. Hardinata, S., Suchyadi, Y.& Wulandari, D. (2021). Strengthening technological literacy in junior high school teachers in the industrial revolution era 4.0. Journal of Humanities and Social Studies, 5(3), 330-335. Higher Education System in Turkey (2018), YOK, Retrieved April 17, 2021 from https://www.yok.gov.tr/en/institutional/higher-education-system History of The Council of Higher Education (2018), YOK, Retrieved April 17, 2021 from https://www.yok.gov.tr/en/institutional/history Hişmanoğlu, M. (2005). Semiotic elements and difficulties in teaching vocabulary items. Dil Dergisi, (128), 51-68. Hixson, N. K., Ravitz, J., & Whisman, A. (2012). Extended professional development in project-based learning: impacts on 21st century skills teaching and student achievement. Charleston, WV: West Virginia Department of Education, Division of Teaching and Learning, Office of Research. Retrieved from https://files.eric.ed.gov/fulltext/ED565466.pdf İstanbul Kültür University ECTS Course Information Pack (2018), Retrieved April 22, 2022 from https://akademikpaket.iku.edu.tr/TR/ects_bolum.php?m=1&p=109&f=12&r=0&ects= ders Ivankova, N. V., Creswell, J. W., & Stick, S. L. (2006). Using mixed-methods sequential explanatory design: From theory to practice. Field methods, 18(1), 3-20. İzmir Demokrasi University ECTS Course Information Pack (2018), Retrieved April 22, 2022 from https://obs.idu.edu.tr/oibs/bologna/index.aspx?lang=tr&curOp=showPac&curUnit=01 &curSunit=15114# Kalonji, G. (2005). Capturing the imagination: High-priority reforms for engineering educators. In National Academy of Engineering (Ed.), Educating the engineer of https://www.gantep.edu.tr/en/ab/dersler.php?bolum=600&bolum_id=604 https://bilsis.hacettepe.edu.tr/oibs/bologna/index.aspx?lang=tr&curOp=showPac&curUnit=440&curSunit=785 https://bilsis.hacettepe.edu.tr/oibs/bologna/index.aspx?lang=tr&curOp=showPac&curUnit=440&curSunit=785 https://www.yok.gov.tr/en/institutional/history https://akademikpaket.iku.edu.tr/TR/ects_bolum.php?m=1&p=109&f=12&r=0&ects=ders https://akademikpaket.iku.edu.tr/TR/ects_bolum.php?m=1&p=109&f=12&r=0&ects=ders Ataberk & Mirici 1542 2020: Adapting engineering education to the new century (pp. 146-150). Washington, DC: National Academies Press Kaplan, D. E. (2019). Creativity in education: Teaching for creativity development. Psychology, 10, 140-147. Karani, A. & Mary, W. M. (2022). Challenges and prospects of online instruction of vocational subjects by TVET institutions in Kenya due to Covid-19. International Journal of Education, Technology and Science (IJETS), 2(2), 108–118. Kivunja, C. (2015) Exploring the pedagogical meaning and implications of the 4Cs “super skills” for the 21st century through Bruner’s 5E lenses of knowledge construction to improve pedagogies of the new learning paradigm. Creative Education, 6(2), 224- 239. Laconi, S., Tricard, N.& Chabrol, H. (2015). Differences between specific and generalized problematic Internet uses according to gender, age, time spent online and psychopathological symptoms. Computers in Human Behavior, 48, 236-244. Lankshear, C., & Knobel, M. (2008). Digital literacies: Concepts, policies and practices. New York: Peter Lang. Ledward, B. C., & Hirata, D. (2011). An overview of 21st century skills. Summary of 21st century skills for students and teachers. Honolulu: Kamehameha Schools–Research & Evaluation Lemke, C. (2002). enGauge 21st Century Skills: Digital Literacies for a Digital Age. Retrieved from: https://www.govinfo.gov/content/pkg/ERIC-ED463753/pdf/ERIC- ED463753.pdf Lewis, A. & Smith, D. (1993) Defining higher order thinking, Theory Into Practice, 32(3), 131-137, DOI: 10.1080/00405849309543588 Lightbown, P. M., & Spada, N. (1999). How languages are learned. Oxford: Oxford University Press. Lima, S.S., Prasetyo, I.& Borotupen Muda,Y.A.(2018) Teachers’ Perception on the Implementation of Knowledge, Skill, Attitude, Value and Ethics (KSAVE) in AllPlus English Language Course and Training. Advances in Social Science, Education and Humanities Research (ASSEHR). 330, 216-220. Maftoon, P.& Amiri, M. (2012) Good Language Learners’ Qualifications and Social - Promotion: A Case Study in the Iranian Context. Journal of Studies in Learning and Teaching English, 1(2), 41-64. Martinez, W.L.& Martinez, A.R. (2001). Computational Statistics Handbook with MATLAB. CRC Press. Boca Raton, Florida. Miller, D.C.& Salkind, N.J. (2002). Handbook of Research Design and Social Measurement. SAGE Publications. Ministry of Education (2022) Fatih Project: Our Vision, Mission& Scope. Retrieved May 31, 2022 from http://fatihprojesi.meb.gov.tr/en/about.html Mirici, İ. H. (2015). Contemporary ELT Practices Across Europe. International Journal of Language Academy- IJLA. 3(4) Winter 2015, 1-8. Moeller& Catalano (2015) Foreign Language Teaching and Learning. International Encyclopedia of the Social & Behavioral Sciences, Second Edition, 327-332. Newby, D. (2012). Insights into the European Portfolio for Student Teachers of Languages (EPOSTL). Cambridge: Cambridge Scholars Publishing Pallant, J. (2016). SPSS Survival Manual: A Step By Step Guide to Data Analysis Using SPSS Program (6th ed.). London, UK: McGraw-Hill Education. Patton, M.Q. (1999). Enhancing the quality and credibility of qualitative analysis. Health Sciences Research, 34, 1189–1208. International Online Journal of Education and Teaching (IOJET) 2022, 9(4), 1513-1544. 1543 P21 Partnership for 21st Century Skills (2009). P21 framework definitions. Retrieved on April 19, 2021 from: http://www.p21.org/documents/P21_Framework_Definitions.pdf P21 Partnership for 21st Century Skills (2019a). Framework for 21st Century Learning, Battelle for Kids. Retrieved in April 19, 2021 from http://static.battelleforkids.org/documents/p21/P21_Framework_Brief.pdf P21 Partnership for 21st Century Skills (2019b). Framework for 21st Century Learning Definitions, Battelle for Kids. Retrieved on April 19, 2021 from http://static.battelleforkids.org/documents/p21/P21_Framework_DefinitionsBFK.pdf Patrick, G. & Esther, C. ed (2015) Assessment and teaching of 21st century skills Springer (Berlin) isbn 978-94-017-9395-7 310 pp €99.99 (ebook €83.29) http://link.springer.com/book/10.1007/978-94-017-9395-7 Paul, R., Binker, A.J.A., & Weil, D. (1990). Critical thinking handbook: K - 3. Rohnert Park, CA: Foundation for Critical Thinking Peters, J. H. (2012). Are they ready? Final year pre-service teachers’ learning about managing student behavior. The Australian Journal of Teacher Education, 37(9), 18- 42. https://doi.org/10.14221/ajte.2012v37n9.2 Price, J., H. & Murnan, J. (2004). Research Limitations and the Necessity of Reporting Them. American Journal of Health Education, 35, 66-67. Promethean. 2017. Digital literacy in the classroom. How important is it? Retrieved from: https://resourced.prometheanworld.com/digital-literacyclassroom-important/ Rasi, P., Vuojärvi, H.& Ruokamo, H. (2019). Media Literacy Education for All Ages. Journal of Media Literacy Education 11 (2), 1 – 19 Rasi, P., Vuojärvi, H., & Rivinen, S. (2021). Promoting Media Literacy Among Older People: A Systematic Review. Adult Education Quarterly, 71(1), 37–54. https://doi.org/10.1177/0741713620923755 Redmond, T. (2016). Learning to teach the media: Pre-service teachers articulate the value of media literacy education in Teacher Education: Concepts, Methodologies, Tools, and Applications (Ed.), 993-1016. Rinaldi, C. (2005). In Dialogue with Reggio Emilia: Listening, Researching and Learning. London: Routledge. http://dx.doi.org/10.4324/9780203317730 Rubin, J. (1975). What the “good language learner” can teach us. TESOL Quarterly, 9 (1), 41-50. Rusdin, N. M. (2018). Teachers’ readiness in implementing 21st century learning. International Journal of Academic Research in Business and Social Sciences, 8(4), 1271–1284. Saavedra, A. R.& Opfer, D. (2012). Teaching and learning 21st century skills: Lessons from the learning sciences. Retrieved April 21, 2021, from http://asiasociety.org/files/rand- 1012report.pdf Saleh, S., E. (2019). 4Cs in the EFL Classroom. 4. 57-80. Saunders, M., Lewis, P., & Thornhill, A. (2012). Research Methods for Business Students, 6th edn, sn. Soulé, H., & Warrick, T. (2015). Defining 21st century readiness for all students: What we know and how to get there. Psychology of Aesthetics, Creativity, and the Arts, 9(2), 178–186. https://doi.org/http://dx.doi.org/10.1037/aca0000017 Spada, N.M. (2007). Communicative Language Teaching. Springer International Handbooks of Education. Stanhope M.& Lancaster, J. (2021) Foundations for Population Health in Community/Public Health Nursing. Elsevier Health Sciences. Stemler, S. (2001). An overview of content analysis, Practical Assessment, Research & Evaluation, 7. 137-146 http://static.battelleforkids.org/documents/p21/P21_Framework_Brief.pdf http://link.springer.com/book/10.1007/978-94-017-9395-7 https://resourced.prometheanworld.com/digital-literacyclassroom-important/ http://asiasociety.org/files/rand-1012report.pdf http://asiasociety.org/files/rand-1012report.pdf https://doi.org/http:/dx.doi.org/10.1037/aca0000017 Ataberk & Mirici 1544 Sural, I. (2017). 21st century skills level of teacher candidates. European Journal of Educational Studies, 3(8), 530-538. Tabachnick, B. G., & Fidell, L. S. (2013). Using Multivariate Statistics (6th ed.). Boston, MA: Pearson. Thornbury, S. (2006) An A-Z of ELT: A dictionary of terms and concepts used in English language teaching. Oxford, UK: Macmillan Education. Tubitak (2021) Telephone: Who invented it? And when? Retrieved May 31, 2022 from https://bilimgenc.tubitak.gov.tr/makale/telefonun- icadi#:~:text=%C4%B0lk%20telefon%201876'da%20Alexander,Martin%20Cooper% 20taraf%C4%B1ndan%20icat%20edildi. Turan, S. (2021) Öğretmenlerin mesleki gelişimi. In Ozdemir, N., Turan, S.& Çoban, Ö. (Eds.), 21.Yüzyıl Okullarını Yeniden Düşünmek (pp.125-141). Ankara: Pegem Akademi. Tyas, M. A., Nurkamto, J., & Marmanto, S. (2020). Cultivating students’ higher order thinking skills in EFL classes: The role of the teacher and the textbook. International Online Journal of Education and Teaching (IOJET), 7(1). 267-276 http://iojet.org/index.php/IOJET/article/view/773 Unal, D.Ç.& Altay, İ.F. (2013) The Effect of Teaching Practicum on Foreign Language Teacher Candidates’ Non-Verbal Communication Use Level. Hacettepe University Journal of Education, 28(3), 417-429 Voogt, J., Erstad, O., Dede, C., & Mishra, P. (2013). Challenges to learning and schooling inthe digital networked world of the twenty-first century. Journal of Computer Assisted Learning, 29, 403–413. Weber R.P. (1990) Basic Content Analysis. Sage Publications, Newburry Park, CA. Yıldız, M.& Palak, D. (2016). Cultivating global competencies for the 21st century classroom: A transformative teaching model. International Journal of Information Communication Technologies and Human Development, 8(1), 69-77. Yost, D. S., Sentner, S. M.& Forlenza-Bailey, A. (2000). An examination of the construct of critical reflection: Implications for teacher education programming in the 21st century. Journal of Teacher Education, 51 (1), 39-49. Yu, L., Recker, M., Chen, S., Zhao, N.& Yang, Q. (2018). The Moderating Effect of Geographic Area on the Relationship Between Age, Gender, and Information and Communication Technology Literacy and Problematic Internet Use. Cyberpsychology, Behavior and Social Networking, 21(6), 367-373. Yüce, E., & Mirici, İ. H. (2019). A qualitative inquiry into the application of 9th grade EFL program in terms of the CEFR. Journal of Language and Linguistic Studies, 15(3), 1171-1187. Živković, S. (2016). A model of critical thinking as an important attribute for success in the 21st century. Procedia - Social and Behavioral Sciences, 232, 102 – 108. Zohrabi, M. (2013). Mixed Method Research: Instruments, Validity, Reliability and Reporting Findings. Theory and Practice in Language Studies, 3, 254-262. http://iojet.org/index.php/IOJET/article/view/773