Tyas, M. A., Nurkamto, J., & Marmanto, S. (2020). Cultivating students’ higher order thinking skills in EFL classes: The role of the teacher and the textbook. International Online Journal of Education and Teaching (IOJET), 7(1). 267-276 http://iojet.org/index.php/IOJET/article/view/773 Received: 03.11.2019 Received in revised form: 12.11.2019 Accepted: 12.12.2019 CULTIVATING STUDENTS’ HIGHER-ORDER THINKING SKILLS IN EFL CLASSES: THE ROLE OF THE TEACHER AND THE TEXTBOOK Case Study Mei Ardaning Tyas Universitas Sebelas Maret meiardaningtyas@gmail.com Joko Nurkamto Universitas Sebelas Maret jokonurkamto@staff.uns.ac.id Sri Marmanto Universitas Sebelas Maret marmanto@staff.uns.ac.id Mei Ardaning Tyas is a student at Graduate Program of English Education Department, Universitas Sebelas Maret, Indonesia. She is interested in TEFL, ESP, and Applied Linguistics. Joko Nurkamto is a lecturer at Graduate Program of English Education Department, Universitas Sebelas Maret, Indonesia. He is interested in TEFL, TESOL, and Linguistics. Sri Marmanto is a lecturer at Graduate Program of English Education Department, Universitas Sebelas Maret, Indonesia. He is interested in Linguistics and Sociolinguistics. Copyright by Informascope. Material published and so copyrighted may not be published elsewhere without the written permission of IOJET. http://iojet.org/index.php/IOJET/article/view/773 mailto:meiardaningtyas@gmail.com mailto:jokonurkamto@staff.uns.ac.id mailto:marmanto@staff.uns.ac.id https://orcid.org/0000-0002-6593-0490 http://orcid.org/0000-0002-8231-0779 https://orcid.org/0000-0002-7198-5404 International Online Journal of Education and Teaching (IOJET) 2020, 7(1), 267-276. 267 CULTIVATING STUDENTS’ HIGHER-ORDER THINKING SKILLS IN EFL CLASSES: THE ROLE OF THE TEACHER AND THE TEXTBOOK Mei Ardaning Tyas meiardaningtyas@gmail.com Joko Nurkamto jokonurkamto@staff.uns.ac.id Sri Marmanto marmanto@staff.uns.ac.id Abstract Incorporating higher order thinking skills (HOTS) in the process of the teaching and learning becomes a demand of the 21st century as well as the latest senior high school curriculum in Indonesia. Hence, this study aimed to know the role of the teachers and the textbooks to promote HOTS in EFL classes. The research was a case study and took place in one of the senior high schools in East Java, Indonesia. The result of the study showed that English language teachers had an adequate understanding about HOTS. However, the teachers were too much dependent on the materials and exercises in the textbooks. In addition, the students’ low level of English language proficiency was the core challenge for them to understand explanations or exercises developed for promoting HOTS of the students. It means that teachers had a crucial role to help their students understand the materials and the exercises presented in the textbooks. To sum up, there was a symbiosis mutualism between English language teachers and the textbooks to promote students’ HOTS. Keywords: Higher-Order Thinking Skills, HOTS, English Teacher, English Textbook 1. Introduction These days, the integration of HOTS in education field is essential since it is one of the fundamental skills in the 21st century skills for the students to be able to face the global challenge (Soulé & Warrick, 2015). The fact that leads the Indonesian Ministry of Education and Culture to incorporate HOTS in the process of teaching and learning through the implementation of the 2013 curriculum (Ahmad, 2014; Setyarini, Muslim, Rukmini, Yuliasri, & Mujianto, 2018), including English subject. As mastering English is crucial in order to have better job prospects, keep up with scientific and technological growth and achieve the academic purpose to access international information (Enç & Kuluşaklı, 2019; Mirici, 2015). However, the practice of HOTS assimilation is not as easy as what has been expected for there are some enormous obstacles faced by education practitioners in the field, especially English teachers (Tyas, Nurkamto, Marmanto, & Laksani, 2019) The aspect which has becomes one of the biggest obstacles in integrating HOTS is related to the teachers’ knowledge and perception towards HOTS. It is crucial for the teachers to understand the concept of HOTS as the right conception will lead to the right practice as well as will contribute to the successfulness of HOTS promotion. Collins (2014) states that teachers’ awareness about the importance of teaching HOTS are very important to guarantee the success mailto:meiardaningtyas@gmail.com mailto:jokonurkamto@staff.uns.ac.id mailto:marmanto@staff.uns.ac.id Tyas, Nurkamto & Marmanto 268 of learning. Nevertheless, the integration of HOTS into teaching-learning activities in the Asian context, including in the Indonesian context, has pros and cons among education stakeholders (Hashim, et al., 2018). Ironically, the study of Yusoff and Seman (2018) indicates that English teachers do not hold sufficient knowledge of HOTS as they could not give satisfactory explanation about the concept of HOTS. In general, Lewis and Smith (1993) define HOTS as the process of taking new information and the information stored in memory and/or reorganizing and encompassing this information to achieve a purpose or find possible answers in novel situations. For an eeducation purpose, Brookhart (2010) claims that HOTS cover three categories, namely transfer, critical thinking, and problem solving. More specifically, Conklin (2012) declares that HOTS encompasses mainly critical thinking and creative thinking. Coincidently, Brookhart (2010) and Conklin (2012) confirm that HOTS in education field are closely linked to Bloom’s revised taxonomy. It is in line with the policy from Ministry of Education and Culture (2018) which classifies HOTS into the cognitive levels, as the parts of Bloom’s revised taxonomy. Nowadays, Bloom’s revised taxonomy, which is set by Anderson and Krathwohl (2001), is currently being used more widely than the original one that was established by Benjamin Bloom. Regarding the cognitive level, the first three levels of cognitive domains in Bloom’s revised taxonomy belong to lower order thinking skills (LOTS), which are remembering (C1), understanding (C2), and applying (C3) whereas, HOTS are categorized by the last three levels of cognitive domain of the taxonomy including analyzing (C4), evaluating (C5), and creating (C6) since the last three levels require different learning and teaching methods than the learning of facts and concepts (Brookhart, 2010; Conklin, 2012; Ministry of Education and Culture, 2018). As the support, Mishra & Kotecha (2016) claim that Bloom's revised taxonomy as a reference point to HOTS. They believe that HOTS is represented in the level of analyzing, evaluating, and creating. Furthermore, Conklin (2012) claims that analyzing (C4) and evaluating (C5) accommodate critical thinking, while creating (C6) represents creative thinking. Conklin (2012) also points out that Bloom’s revised taxonomy is a viable questioning strategy that can promote HOTS in the classroom. Hence, the taxonomy can be used by teachers to plan instructions in order to facilitate the teaching and learning of HOTS, as well as to help students develop their thinking skills. Even though it is clear that Bloom’s revised taxonomy can be used by the teachers as a questioning strategy that can promote HOTS in the classroom, the study of Sunggingwati and Nguyen (2013) shows that the Indonesian English teachers rely on the textbook for pedagogies and for the kinds of questions that they ask to the students. The teachers are exposed mainly to low-level questions. Thus, they face some challenges in generating high-level questions in these conditions and require assistance, especially in the form of a textbook for guidance in order to do this. Textbooks belong to the main material sources in EFL teaching and learning process in Indonesia. Notwithstanding the ongoing reputation of computer-assisted and multimedia-enhanced English instruction with the advancement of English teaching reform, textbooks remain the core of teaching and testing (Yidi & Zhengwei, 2018). Consequently, the government needs to provide a well-designed English textbook to help the teachers in developing good instructions. The textbook is hoped to provide teachers with teaching ideas and plans for teaching English in the classroom. It has to contain HOTS – based exercises as what has been mandated by the 2013 curriculum in order to facilitate teachers and students in the teaching and learning of HOTS. Furthermore, textbooks could be one of the best and most reliable sources for teachers to teach and for students to learn how to be higher-order thinkers (Tarman & Kuran, 2015). International Online Journal of Education and Teaching (IOJET) 2020, 7(1), 267-276. 269 Unfortunately, some of the studies which examine the questions in English textbook show that most questions found in English textbooks prescribed for students, are in LOTS levels. Maharani, et al. (2018) conducts an evaluative research to find out the quality of the English textbook for tenth grade students published by the Ministry of Education and Culture in term of instruction used. The result shows that the textbook is not qualified as the good textbook because there are found only 40% instructions which belong to HOTS – based instructions from 234 instructions. The same phenomena also found in the Asian context in which the questions contained in English textbooks are mostly aligned with LOTS when analyzed using Bloom’s revised taxonomy (Anasy, 2016; Olimat, 2015; Raqqad & Ismail, 2018; Ulum, 2016; Zareian, Davoudi, Heshmatifar, & Rahimi, 2015). Otherwise, Yuliana & Tungka (2018) who examine the critical thinking in the reading section in the government English textbook for Indonesian eleventh grades of senior high school find that the HOTS – based exercises proportion is bigger than LOTS – based exercises proportion. It means that the examined textbook meets the criteria of a good textbook as it contains a greater proportion of HOTS – based questions (Sunggingwati & Nguyen, 2013). In other words, it can be said that the examined English textbook can facilitate the teachers in teaching English as well as in developing students’ HOTS. In spite of the inadequacy of English teachers and English textbooks, this article aims to explore the role of English teachers and English textbooks in cultivating students’ HOTS through English language teaching. 2. Methodology In this qualitative research, a descriptive case study design was used. It was intended to make the researchers able to analyze the case or phenomenon which happens in the field. In addition, Yin (2018) states that the main goal of the descriptive case study is to assess a sample in detail and in-depth, based on an articulation of a descriptive theory. The research was conducted in one of the state senior high schools in East Java, Indonesia. The school was chosen since it belongs to the school which has implemented the pilot project of the 2013 curriculum which means that HOTS is already integrated in the teaching and learning process in this school. An English teacher of the eleventh grade is selected as the participant of this research as it is related to the existence of the government English textbook for the eleventh grade in which it contained a good proportion of HOTS – based exercises (Yuliana & Tungka, 2018). Questionnaire, observation, and interview were used to collect the data. A semi-structured questionnaire was given to the teacher in order to know her knowledge and understanding towards HOTS. In addition, non-participatory observation in the classroom was held to know the role of English teacher and English textbook being used in cultivating students’ HOTS. To ensure the data which were obtained from questionnaire and observation, a semi-structured interview was also carried out with the English teacher. For obtaining in-depth information about the role of the English teacher and English textbook to promote students’ HOTS, a structured questionnaire was given to the students of eleventh grade who are taught by the English teacher being investigated. A semi-structured interview was also held with the students to get the detail information of the case. The data of this research were in the form of information related to the role of English teacher and English textbook to cultivate students’ HOTS. To analyze the data gotten from the field, a data analysis technique from Miles, Huberman,and Saldana (2018). The steps of analyzing the data are data condensation, data display, and data conclusion. The process of data condensation included writing summaries, coding, developing themes, generating categories, and writing analytic memos. After that, the result of questionnaire, observation, and interview which had been proceed in the data condensation was displayed descriptively so that the Tyas, Nurkamto & Marmanto 270 information related to the research’s issue could be seen evidently. The last step in data analysis process is drawing conclusion in which the data were verified and the initial conclusion was made. 3. Findings and Discussion The findings of the research as well as the discussion are presented under the subtitle in accordance with some important aspects as follows: 3.1. The English Teacher’s Understanding and Perception towards HOTS Knowing the English teacher’s understanding and perception towards HOTS is pivotal in the process of promoting students’ HOTS. It is due to the fact that the understanding and perception of the teacher towards HOTS will affect the successfulness of HOTS cultivation. Further, experts claim that the right conception of HOTS that the teacher has will lead to the right practice as well as will contribute to students’ HOTS development. The background of the teacher was also the important point to be portrayed. As what had been stated before that the research was taken place in the state senior high school which belong to the pilot school of the 2013 curriculum enactment, it could be understood that the teacher was already familiar with the integration of HOTS in the teaching and learning process. Moreover, based on the teacher’s recognition, she had joined some workshops and trainings related to the 2013 curriculum implementation as well as HOTS integration. According to the teacher’s explanation, HOTS are closely connected to the use of critical thinking and creative thinking. She also mentioned about the cognitive level of Bloom’s revised taxonomy but she did not state further about the connection of critical thinking and creative thinking with the taxonomy. Additionally, she said that the three top cognitive levels in the taxonomy, which are analyzing (C4), evaluating (C5), and creating (C6) belong to HOTS. The explanation of the teacher is in line with the statement of Brookhart (2010) and Conklin (2012) about the use of Bloom’s revised taxonomy as a guidance to develop students’ HOTS. Being asked further about the integration of HOTS in the process of English teaching and learning, she claimed that HOTS could be presented to the students through HOTS – based exercises and HOTS – based tasks. When being asked about the kind of questions which belong to HOTS, she answered that the questions which belong to HOTS are the difficult questions. The result showed that the teacher still misinterpreted the concepts of HOTS. Further, it can be seen that she is still confused about differentiating skills for solving HOTS problems and skills for solving difficult problems. She had a tendency to assume that HOTS – based questions are the questions which categorized as difficult and complex, whereas, indeed, HOTS – based questions are not always the difficult one, and vice versa (Brookhart, 2010; Conklin, 2012; Mishra & Kotecha, 2016). In addition, Tyas et al., (2019) ensure that the precise knowledge of HOTS is the English teachers’ rudimentary armament in promoting students’ HOTS through the process of teaching and learning. It means that holding the right conception of HOTS will give advantages for English teachers in cultivating students’ HOTS. Talking further about the teacher’s perception towards HOTS, she welcomed the integration of HOTS very much and had a willingness to integrate HOTS in the teaching-learning process since she realized that the skills are needed by the students to face the challenges of the 21st century. However, she explained that she found some difficulties to promote students’ HOTS, among others is the heterogeneity of students’ thinking ability as well as language mastery. In more detail, she explained that giving HOTS – based questions or exercises to the students whose English mastery is in the upper intermediate level is much easier than giving the questions or exercises to them whose English mastery is in the lower intermediate level. The students with lower intermediate level of English mastery will deal with two things at the same International Online Journal of Education and Teaching (IOJET) 2020, 7(1), 267-276. 271 time, they are the language barriers and the thinking level as well for English is a foreign language for Indonesian students. Related to the existence of English as a foreign language teaching, the study of Singh et al., (2018) indicates that weak English as foreign language learners need special treatment in the teaching and learning process which means that the teacher is required to give more attention to them. From the result of the English teacher’s understanding and affection towards HOTS, it could be portrayed that the teacher is aware of the importance of HOTS and has a strong willingness to integrate HOTS in her classroom. In addition, she actually holds a good understanding about the general concept of HOTS in which she could explained the definition as well as the characteristics of HOTS well. She also has adequate understanding about the relationship between HOTS and Bloom’ revised taxonomy. Yet, she still finds difficulties in distinguishing skills for solving HOTS problems and skills for solving difficult problems. 3.2. The Role of English Teacher and English Textbook to Cultivate Students’ HOTS After knowing the understanding and perception of the English teacher towards HOTS, it seemed easier to know the teacher’s role in cultivating students’ HOTS. Through the classroom observation, it is known that the teacher started the lesson by giving some initial questions about the material that would be delivered. The questions that the teacher gave were about to engage the students to the material as well as to connect their real life with the material. The material that was given is exposition text. However, the questions given by the teachers were almost categorized as LOTS – based question, such as: Table 1. Questions produced by the English teacher No. Questions Types 1. What do you know about bullying? Lower order thinking skills (LOTS) 2. Have you ever experienced bullying? Lower order thinking skills (LOTS) The questions mentioned above are belong to LOTS – based question in which the questions only focus to know the students’ understanding towards the topic given. After giving some initial questions about the topic, the teacher asked to students to look at certain page of the textbook that has the same topic with the previous questions the teacher gave, that is bullying. In the textbook, a text about bullying and a set of questions are presented well. The students were asked to read the text consecutively, after that they were asked to answer some questions following the text in the textbook individually, as follows: Table 2. Questions from the English textbook for individual work No. Questions Types 1. Do you think bullying is a serious issue in your school? Give reasons to support your opinion. Higher order thinking skills (HOTS) 2. Did this opinion article raise/change your awareness about bullying? Please explain. Higher order thinking skills (HOTS) 3. Do you think it is necessary to educate people on the issue of bullying? Why? Give reasons to support your opinion. Higher order thinking skills (HOTS) 4. Do you think bullying should be declared as a punishable crime? Give reasons to support your answer. Higher order thinking skills (HOTS) Having answered some questions in the textbook individually, the teacher then divided the class into several groups to have a discussion. The discussion was initiated by the questions presented in the textbook, as follows: Tyas, Nurkamto & Marmanto 272 Table 3. Questions from the English textbook for group work No. Questions Types 1. Discuss the article on bullying in a group. Does it change your perspective on bullying or not? Give reasons to support your answer. Higher order thinking skills (HOTS) 2. Are you aware of cyber bullying? Do you think it is worse than physical bullying? Why? Support your opinion with examples. Higher order thinking skills (HOTS) 3. What can young people like you do to prevent or stop bullying? List at least three things you and your friends can do to prevent or stop bullying. Higher order thinking skills (HOTS) By looking at the set of questions from the textbook, it is clear that the questions are belong to HOTS – based questions since the questions triggered the students to take and use information as a tool to find possible answers or to solve problems in a new situation. It is in line with the definition of HOTS which is proposed by Lewis and Smith (1993). Moreover, the questions are classified into HOTS as the questions are belong to the level of analyzing (C4) and evaluating (C5). In the end of the lesson, the teacher gave the students a homework to write a personal journal in which the question was taken from the textbook, as follows: Table 4. Questions from the English textbook for homework No. Questions Types 1. Place yourself in the shoes of a person who is bullied every day at school. How would you feel? Write down your feelings and what would you do about it? Higher order thinking skills (HOTS) The question for the homework is also classified as HOTS – based questions since it represents the level of creating (C6) of Bloom’s revised taxonomy. From the result presented above, it could be clearly seen that the questions from the textbook are more HOTS oriented than the questions produced by the teacher. It means that, by looking at the classification of the questions, the questions contained in the textbook have more possibility to cultivate students’ HOTS than the teacher’s questions. Moreover, the result of the observation showed that the teacher tended to depend a lot on the materials and exercises contained in the textbook. Hence, Shomoossi (2004) suggests English teachers to use the questions or exercises provided in textbooks rather than generate questions themselves regarding the types of the questions. 3.3. The Students’ responses towards the Role of English Teacher and English Textbook to Cultivate Students’ HOTS In the process of investigating the role of English teacher and English textbook to cultivate students’ HOTS, it is crucial to know the students’ responses since they are the parties who have direct contact with the teacher and the textbook, and their responses would be beneficial to search in-depth information as well. In the process of collecting students’ responses, the researchers tried to group the students based on their language mastery; they are above and below the average levels. The grouping process is intended to investigate each group response as the teacher said that the heterogeneity of the students’ language mastery becomes the biggest obstacle along with students’ cognitive level diversity. International Online Journal of Education and Teaching (IOJET) 2020, 7(1), 267-276. 273 The students who have above the average level of English mastery were given a questionnaire which consisted of fifteen statements in which they are asked to give yes/no response towards the statement. The statements are about the use of the question in the textbook, the teacher’s role, as well as the process of the teaching and learning in developing their HOTS. After getting the students’ responses from the questionnaire, an interview was also conducted to clarify their responses. The students claimed that they are already familiar with the integration of HOTS in English lesson; they also stated that it is the part of their routine English lesson. Being asked about the questions/exercises contained in the textbook that they use as learning material in the classroom, they explained that they did not find any meaningful challenges in understanding and answering the questions presented. When shown questions which belong to HOTS – based questions from the textbook, they showed the same response that they could answer the questions well. In the process of the teaching and learning, the students showed their enthusiasm in following the lesson, yet they did not asked questions to the teacher. When the researchers wanted to clarify the reason behind it, the students argued that they have understood the meaning of the questions and did not need any further explanation about the questions. Talking about the role of the teacher in cultivating their HOTS, the students explained that the teacher plays a bigger role in presenting to them the material related to lesson and has a smaller role in developing their HOTS. They claimed that the questions in the textbook contributed to the cultivation of their HOTS. Moreover, in understanding and answering the questions related to HOTS, they do not find any meaningful difficulties. The same questionnaire was given to the students whose English mastery is below the average level, an interview was also held to search for in-depth responses. The students had the same claim with the students from the group of above the average English mastery level in which they are already familiar with the integration of HOTS as it is the part of the routine English teaching and learning in their classroom. Being asked about the questions/exercises presented in the government English textbook for the eleventh graders, they admitted that they have difficulties in understanding the meaning of the questions. Their responses are understandable since it is closely related to their language mastery and their lack of vocabulary mastery. Having difficulties to understand the meaning of the questions affects their ability in solving or answering the questions presented to them. The problem makes them face obstacle in solving or answering the questions. The similar response was given by them when the researchers showed them HOTS – based questions contained in the textbook. The response from the students proves that the language barriers do exist. It is in line with the answer of the teacher in which she claimed that language barriers become the obstacle in developing students’ HOTS. Due to their limited vocabulary mastery, they often asked questions to the teacher in the process of teaching and learning, their questions were about the meaning of words that they do not know. Interestingly, the teacher did not answer the students’ questions directly, instead she tried to give clues or hints related to the meaning of the words the students did not know. She allowed the students to open the dictionary but it is an English to English dictionary in which the definition of the words they asked are also presented in English, this fact required the students to search for clearer answers from the teacher. Talking about the role of the teacher in cultivating their HOTS, the students admitted that she plays a pivotal role in the process of developing their HOTS. It is related to the teacher’ role in helping them understand and solve the questions or problems which belong to HOTS type. Further, they claimed that the teacher plays an important role in explaining them the material related to lesson, helping them to understand and solve or answer HOTS – based questions. Specifically, they stated that the helps from the teacher enable them to understand and solve or answer HOTS – based questions although the teacher are rarely giving them HOTS – based questions from her own. They explained that even though the textbook contained HOTS – based questions, they could not be higher order thinker if they do not understand the meaning of HOTS – based Tyas, Nurkamto & Marmanto 274 questions contained on it. Moreover, the way the teacher answers their questions, especially about giving clues or hints related to the meaning of words they do not know the meaning, spurs them to think critically. In the end, they said that the teacher has a bigger role in cultivating their HOTS than the role of the questions contained in the textbook. From the result of students’ responses towards the role of English teacher and English textbook, it can be seen that there is a different response between the students whose English mastery is above the average and the students whose English mastery is below the average level. The students with above the average English proficiency claimed that the textbook has a bigger role to cultivate their HOTS, while the students with under the average English proficiency admitted that the teacher plays a more instrumental role than the textbook in promoting their HOTS based on the reasons explained before. 4. Conclusion Regarding the role of English teacher and English textbook in cultivating students’ HOTS through language teaching, it can be concluded that based on the types of the questions presented, the questions contained in the textbook are more HOTS oriented than the questions produced by the teacher in which it means that the textbook has bigger possibility to promote students’ HOTS. In addition, the result of the observation showed that the teacher tended to depend a lot on the material and questions contained in the textbook. However, the teacher also plays an instrumental role in helping the students with language barrier in the process of understanding and answering HOTS – based questions contained in the textbook. It means that there is a symbiotic mutualism between English teacher and English textbook in cultivating students’ HOTS. Further, the students with language barriers claimed that the teacher helps them a lot to be higher order thinker. Based on the result of this research, it also could be seen that although the teacher has an adequate understanding about HOTS, although she still holds finds difficulties in distinguishing skills for solving HOTS problems and skills for solving difficult problems. It is the responsibility of the Indonesian government through the Ministry of Education and Culture to give workshops, trainings, and evaluations to ensure that the education practitioners, especially teachers, hold the right understanding and concepts of HOTS in order to be able to integrate it in the teaching and learning process. The teacher’s dependency on the materials and exercises contained in the textbook to cultivate students’ HOTS is a main trigger for Indonesian government to develop textbooks which can accommodate teachers in the process of HOTS promotion. It also provides a strong basis to conduct future research on the importance of content analysis of the textbook, especially the types of the exercises contained in the English textbooks which are used as main materials in the teaching and learning process. It is also suggested to conduct the research about the implementation of the exercises contained in textbooks in promoting students’ HOTS. International Online Journal of Education and Teaching (IOJET) 2020, 7(1), 267-276. 275 References Ahmad, D. (2014). Understanding the 2013 curriculum of English teaching through the teachers’ and policymakers’ perspectives. International Journal of Enhanced Research in Educational Development (IJERED), 2(4), 6–15. Anasy, Z. (2016). HOTS (Higher Order Thinking Skills) in reading exercise. Tarbiya: Journal of Education in Muslim Society, 3(1), 51–63. https://doi.org/https://doi.org/10.15408/tjems.v3il.3886 Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Addison Wesley Longman, Inc. Brookhart, S. M. (2010). How to assess higher-order thinking skills in your classroom. Alexandria: ASCD. Collins, R. (2014). Skills for the 21st Century: teaching higher-order thinking. Curriculum & Leadership Journal, 12(14). Conklin, W. (2012). Higher-order thinking skills to develop 21st century learners. Shell Education Publishing, Inc. Enç, G., & Kuluşaklı, E. (2019). The evaluation of language teaching program applied in the school of foreign languages. International Online Journal of Education and Teaching (IOJET), 6(3), 658–670. Retrieved from http://iojet.org/index.php/IOJET/article/view/631 Hashim, H., Ali, M. N., & Shamsudin, M. A. (2018). Infusing high order thinking skills (HOTs) through thinking based learning (TBL) during ECA to enhance students interest in STEM. International Journal of Academic Research in Business and Social Sciences, 7(11), 1191– 1199. https://doi.org/10.6007/ijarbss/v7-i11/3557 Lewis, A., & Smith, D. (1993). Defining higher order thinking. Theory into Practice, 32(3), 131–137. https://doi.org/10.1080/00405849309543588 Maharani, S., Rozani, S. A. F., & Fatimah, S. (2018). AN EVALUATION ON THE INSTRUCTIONS IN THE ENGLISH. E-Journal of English Language and Literature, 7(3), 520–530. Miles, M. B., Huberman, A. M., & Saldana, J. (2018). Qualitative data analysis: A methods sourcebook (4th ed.). SAGE Publications. Ministry of Education and Culture. (2018). Buku Pegangan Pembelajaran Berorientasi pada Keterampilan Berpikir Tingkat Tinggi. Jakarta: Ministry of Education and Culture. Mirici, I. H. (2015). A digital european self-assessment tool for student teachers of foreign languages : The EPOSTL. The Turkish Online Journal of Educationan Technology - TOJET, 14(1), 1–10. Mishra, R., & Kotecha, K. (2016). Are we there Yet! Inclusion of higher order thinking skills (HOTs) in assessment. Journal of Engineering Education Transformations, 2–5. https://doi.org/10.16920/jeet/2016/v0i0/85686 Olimat, M. (2015). Analyzing action pack textbooks’ questions according to revised Bloom taxonomy. Journal of Education and Practice, 6(28), 152–159. Raqqad, Y. Al, & Ismail, H. H. (2018). Analyzing the reading questions of ap12 textbook according to analyzing the reading questions of ap12 textbook according to Bloom’s taxonomy. International Journal of Education, Psychology and Counseling, 3(22), 84–94. Tyas, Nurkamto & Marmanto 276 Setyarini, S., Muslim, A. B., Rukmini, D., Yuliasri, I., & Mujianto, Y. (2018). Thinking critically while storytelling: Improving children’s HOTS and English oral competence. Indonesian Journal of Applied Linguistics, 8(1), 189–197. https://doi.org/10.17509/ijal.v8i1.11480 Shomoossi, N. (2004). The effect of teachers’ questioning behavior on EFL classroom interaction: A classroom research study. The Reading Matrix, 4(2), 96–104. Singh, C. K. S., Singh, R. K. A., Singh, T. S. M., Mostafa, N. A., & Mohtar, T. M. T. (2018). Developing a higher order thinking skills module for weak ESL learners. English Language Teaching, 11(7), 86. https://doi.org/10.5539/elt.v11n7p86 Soulé, H., & Warrick, T. (2015). Defining 21st century readiness for all students: What we know and how to get there. Psychology of Aesthetics, Creativity, and the Arts, 9(2), 178– 186. https://doi.org/http://dx.doi.org/10.1037/aca0000017 Sunggingwati, D., & Nguyen, H. T. M. (2013). Teachers’ questioning in reading lessons: A case study in Indonesia. Electronic Journal of Foreign Language Teaching, 10(1), 80–95. Tarman, B., & Kuran, B. (2015). Examination of the cognitive level of questions in social tudies textbooks and the views of teachers based on Bloom Taxonomy examination of the cognitive level of questions in social studies textbooks and the views of teachers based on Bloom Taxonomy. Educational Sciences: Theory & Practice, 15(1), 1–10. https://doi.org/10.12738/estp.2015.1.2625 Tyas, M. A., Nurkamto, J., Marmanto, S., & Laksani, H. (2019). Developing higher order thinking skills ( HOTS ) – Based questions : Indonesian EFL teachers ’ challenges. Proceedings of the International Conference on Future of Education, 2(1), 52–63. https://doi.org/https://doi.org/10.17501/26307413.2019.2106 Ulum, Ö. G. (2016). A Descriptive content analysis of the extent of Bloom ’ s Taxonomy in the reading comprehension questions of the course book Q : Skills for Success 4 Reading and Writing. The Qualitative Report, 21(9), 1674–1683. Yidi, W., & Zhengwei, P. (2018). An investigation of critical thinking manifested in the questions of EFL textbooks for tertiary-level English majors of China. American Journal of Education and Learning, 3(2), 72–84. https://doi.org/10.20448/804.3.2.72.84 Yin, R. K. (2018). Case study research and applications: Designs and methods (6th ed.). Los Angeles: SAGE. Yuliana, S., & Tungka, N. F. (2018). Critical thinking questions in the reading section of EFL textbooks. Alphabet, 1(1), 1–12. https://doi.org/10.21776/ub.alphabet.2018.01.01.01 Yusoff, W. M. W., & Seman, S. C. (2018). Teachers’ knowledge of higher order thinking and questioning skills: A case study at a primary school in Terengganu, Malaysia. DEVELOPMENT, 7(2). Zareian, G., Davoudi, M., Heshmatifar, Z., & Rahimi, J. (2015). An evaluation of questions in Two ESP coursebooks based on Bloom’s New Taxonomy of cognitive learning domain. International Journal of Education and Research, 3(8), 313–326.