Microsoft Word - Surono 240-252.docx English Language Teaching Educational Journal ISSN 2621-6485 Vol. 5, No. 3, December 2022, pp. 240-252 https://doi.org/10.12928/eltej.v5i3.8168 http://journal2.uad.ac.id/index.php/eltej/index eltej@pbi.uad.ac.id Analysis of HOTS and LOTS of instructional questions in the English textbook "When English Rings a Bell" for grade VIII Surono a,1*, Bambang Widi Pratolob,2, Siti Latifah Hanun c,3 a, b, c Universitas Ahmad Dahlan, Jl. Ringroad Selatan, Kragilan, Tamanan, Kec. Banguntapan, Kabupaten Bantul Daerah Istimewa Yogyakarta 55191 1 surono@pbi.uad.ac.id, 2bambang.pratolo@pbi.uad.ac.id, 3sitilatifahhanun669@gmail.com * Corresponding author A R T I C L E I N F O A B S T R A C T Article history Received 13 September 2022 Revised 25 October 2022 Accepted 5 December 2022 English textbooks have an essential role in learning English subjects for students in junior high school, so the quality of textbooks must be of high quality. To make students master English language skills, they need a good quality of textbooks to improve their critical thinking skills. This research aims to find out the representation of higher order thinking skills (HOTS) and lower order thinking skills (LOTS) and the appearance frequency in the instructional questions in English textbook entitled “When English Rings A Bell” grade VIII. This study used a content analysis with qualitative approach. Six chapters of the textbook were taken as the sample. Observation and note-taking techniques were used to collect the data using observation sheet. The data were analyzed by using Miles’s et al. model regarding coding method. The research findings showed that chapters I, VI, and VII represented LOTS with the most dominant in remembering and applying. Meanwhile, the appearance of LOTS and HOTS in chapters V and XII looked nearly in balance dominating the verb remembering for LOTS and the verbs evaluating and creating for HOTS. The realization of HOTS in the chapters was less than LOTS, except chapter XIII. After the instructional questions in all activities were categorized, the distribution of LOTS in the textbook appeared in 66 (58%) and HOTS in 47 questions (42%). It is concluded that the textbook distributed less evenly in HOTS and LOTS composition. It implies that the textbook still focuses on LOTS. It is recommended that the authors of the textbook balance the HOTS and LOTS in the instructional questions proportionally to encourage students to think critically. Teachers should creatively modify the instructional questions to meet the balance. This is an open access article under the CC–BY-SA license. Keywords HOTS LOTS Instructional Questions Content Analysis Textbook How to Cite: Surono, Pratolo. B. W., & Hanun, L. S. (2022). Analysis of HOTS and LOTS of instructional questions in the English textbook “When English Rings a Bell” for grade VIII. English Language Teaching Educational Journal, 5 (3), 240-252. https://doi.org/10.12928/eltej.v5i3.8168 1. Introduction Nowadays, education must go beyond simply retaining material and memorization (Qasrawi & BenyAbdelrahman, 2020). Higher order thinking skills (HOTS) and lower order thinking skills (LOTS) are terms used in education. The Indonesian government popularized HOTS, and the idea was used in the country's national test (Ariawan et al., 2023). HOTS are very often incorporated in instructional questions of a textbook. The purpose of instructional questions is to get students to 241 English Language Teaching Educational Journal ISSN 2621-6485 Vol. 5, No, December 2022, pp. 240-252 Surono et.al (Analysis of LOTS and HOTS of instructional questions……) reflect on and interact with the subject matter. Instructional questions are frequently included in textbooks to aid students in understanding and applying the principles they are learning including the culture of the target language (Soviyah, 2022). Depending on how much thought is needed to respond to each topic, they can either be classified as HOTS or LOTS. However, higher cognitive levels are necessary to increase students' awareness of their own learning processes and teach them how to use such capacities to improve their academic performance. (Ching & da Silva, 2017). One of the most crucial and integral parts of the language instruction process is the use of textbooks (Dabbagh & Safai, 2019). HOTS development resources and exercises are a requirement of a good textbook (Erdiana & Panjaitan, 2023). So, it is essential to develop textbooks with adequate material in order to fulfill learning objectives (Lestari et al., 2022). Wale and Bogale (2021) stated that textbooks and the educational field are intricately related. A textbook is a piece of educational material that is used in schools and universities to support a teaching and learning program, based on the modern and widely accepted definition. However, the teachers’ expertise is very crucial to complete the role of textbook (Apichat & Fatimah). Additionally, teachers must have the ability to modify the learning paradigm to fit the circumstances, including in selecting the textbook (Purwanti et al., 2023). In the teaching and learning process, textbooks can be used as a guide for teachers and students, namely as the main reference or as a supplementary material. In learning activities, students are not limited to observing what is explained by the teacher. Students also need references to explore knowledge so that their abilities and critical thinking can be optimized. According to Febrina et al. (2019), reading a suitable textbook with HOTS issues is essential for students to strengthen their critical thinking abilities (Hatmanto et al., 2023), which assist students in navigating constraints and successfully resolving problems (Sutama et al., 2022). Thus, teachers must intelligently determine which textbooks should be used in the learning process. If a teacher can determine a good textbook, it will consequently have a big influence in the learning process (Tomlinson & Masuhara, 2018). English assignments from textbooks are crucial to learning. Because of this, instructional questions give students the chance to hone their abilities and produce effective teaching and learning. By practice, language learning exercises are streamlined and completely grasped. Students are required to complete instructional questions from textbooks as one of their exercises (Gilakjani & Sabouri, 2016). Because "When English Rings A Bell" is a guide book that highlights the potential to build information, abilities, and attitudes as a complement and companion for learning English, the researchers chose it as the material for the study. This textbook, which is a component of the advanced 2013 curriculum, exhorts students to engage in active learning and critical thinking up until the production stage. (Errington & Bubna-Litic, 2015). English textbooks, for example, which are skill-based, are seen as a rich source of resources and content that may represent the HOTS and LOTS, due to the large number of reading passages and writing activities (Qasrawi & BenyAbdelrahman, 2020). Unfortunately, many textbooks do not represent HOTS proportionally and only focus on LOTS (Hasanah, 2017). In fact, students must also be familiar with the HOTS materials they are learning. Therefore, the representation of HOTS and LOTS should be proportional in the English textbook. This is because incorporating HOTS in textbooks can help students understand critical thinking. In turn, this can help teachers to be able to advance their students through Bloom's Taxonomy levels (Kelly, 2014). Critical thinking requires creativity, careful analysis, and the ability to adapt (Changwong et al., 2018). There have been many researches on textbook analysis with different focuses and intensities. Sucipto and Cahyo (2019) analyzed the reading assignments in the English textbook "Bright 2". Similarly, Azizah and Bharati (2021) explored reading activities in Package B modules. Then, Similarly, Febriyani et al. (2019), Nastiti (2020), Fahmi (2020), Aryani and Wahyuni (2020), Ariawan et al. (2023), and Erdiana and Panjaitan (2023) analyzed the HOTS focusing on the reading materials from English textbook. Zainil et al. (2019) also looked into the HOTS reading comprehension problems found in the English textbooks for a senior high school in Padang. Rindawati, et al. (2014) analyzed writing skill in English textbooks to improve students' positive attitudes and communicative competence. Meanwhile, Zaiturrahmi, et al. (2017) focused their analysis on the exercises of English skills with limited varieties in the textbook entitled "When English Rings a Bell" for class X. They also revealed HOTS and LOTS in the exercises in ELT ISSN 2621-6485 English Language Teaching Educational Journal 242 Vol. 5, No, December 2022, pp. 240-252 Surono et.al (Analysis of LOTS and HOTS of instructional questions……) textbooks, and they focused more on discussing textbook quality. However, they only focused on different aspects and language skills in the textbook. The same thing happened to HOTS they analyzed in instructional questions which tended to only discuss one English skill. Though analyzing HOTS, Rizkiani (2022) just focused on the integration of HOTS in lesson objectives for elementary school textbook. In more general idea, the idea of fostering critical thinking in speaking was discussed (Viana et al., 2022), writing (Arifin et al., 2020), and listening abilities (Irianti et al., 2022). A similar study but different in viewpoint was conducted by Dabbagh and Safai (2019). They analyzed Iranian nationwide ELT textbooks. Then, Kamarulzaman et al. (2017) compared HOTS and LOTS scores gained by the students. In sum, the previous studies explored HOTS and or LOTS in English skills separately. However, though in a bit contrast, this study focused on discussing HOTS and LOTS in each instructional question contained in all skills altogether in the textbook. Based on the description above, two problems were formulated as the main discussion. They were (1) How is the representation of HOTS and LOTS in the instructional questions in the English textbook “When English Rings a Bell”? and (2) How frequently are HOTS and LOTS emphasized in the activities in the English textbook? These two issues are very important to investigate because the EFL textbooks published by the Indonesian government are used by English teachers throughout Indonesia, and thus are expected to improve the quality of the instructional questions contained in the textbooks. This study aims to analyze the representation and frequencies of instructional questions considering the HOTS and LOTS. In terms of cognitive domains, the revised Bloom’s taxonomy (Anderson et al., 2001) indicates that there are two categories of cognitive domains that must be included in the textbook, namely HOTS and LOTS. In LOTS, what students do is usually in the form of taking notes, copying, memorizing, or following. In other words, LOTS include remembering (C1), which refers to the ability to retrieve, recall, or recognize pertinent knowledge from long-term memory; students may mention definitions, imitate pronunciations, state structures, pronounce them, and repeat; understanding (C2), which requires students to explain an idea, principle, rules, or practice in one or more ways to show that they have understood; and applying (C3), that is, students can apply their understanding in a new scenario by employing information or a skill in a new situation HOTS is defined as the ability to think strategically to use information in solving problems, analyzing arguments, negotiating issues, or making predictions. It can be concluded that the understanding of HOTS is a complicated level thinking ability that requires the use of description, conclusion, analysis, and other higher order thinking skills and actual instances. Anderson et al. (2001) states that HOTS include analyzing (C4), namely the skill of being able to break down concepts into their component parts and connect them in order to fully understand the concept; evaluating (C5) is the ability to determine the degree of something based on certain norms, criteria or benchmarks; and creating (C6), namely the ability to integrate pieces into a new whole and broad form, or to create something original. Therefore, it is very important to design textbooks with appropriate content of LOTS and HOTS so that learning objectives can be achieved. The implication of this research is that it can provide valuable information to the stakeholders involved in learning in the classroom, curriculum design, and syllabus development. In addition, materials development also gets teachers to discover how to better overcome the textbook's weaknesses with innovative methods to achieve the learning goals effectively (Hafizhah & Pratolo, 2022). 2. Method This descriptive qualitative study employed content analysis to describe the information obtained from analyzing a textbook. The research subject of this research was the English textbook "When English Rings A Bell" for grade VIII published by Ministry of Education and Culture of the Republic of Indonesia which consisted of 13 chapters with 234 pages in total. The data of this research were gathered through observing six chapters selected to represent other chapters and note taking on the instructional questions in the English textbook. An observation sheet was used as the instrument to collect data which was based on revised Bloom’s taxonomy (Anderson et al., 2001) to observe the representation of HOTS and LOTS in the instructional questions. Then, the instructional questions and activities contained in each of the six chapters in the textbook were listed. In order to obtain valid and reliable data, the researchers conducted a data trustworthiness using four criteria, namely credibility, transferability, dependability and confirmability (Elo et al., 2014). 243 English Language Teaching Educational Journal ISSN 2621-6485 Vol. 5, No, December 2022, pp. 240-252 Surono et.al (Analysis of LOTS and HOTS of instructional questions……) To analyze the data, several steps were taken based on Miles’s et al, (2004) approach by regarding coding method (St. Pierre & Jackson, 2014; Schilling, 2006). In data reduction, two steps were conducted, namely classifying and coding. The classification in this study was based type of cognitive domains – HOTS or LOTS. So, to make it easier to classify the HOTS and LOTS in textbooks, each instructional question was coded. After that, the first data display step was to make an observation sheet assist researchers to find out the number of each type of HOTS and LOTS. After the data were calculated, they were input in a table. The data on each page were compiled into a chapter, and each chapter was compiled into the entire book. Then, the percentage of the data were presented in tables to see the difference in the number between the HOTS and LOTS of instructional questions. Finally, after all the analysis processes were carried out and described, the results of the analysis were concluded. 3. Findings and Discussion This study set out to find out the representation and frequencies of HOTS and LOTS in instructional questions of all the English skills. The following discussion focuses on the analysis of the instructional questions from textbook regarding HOTS and LOTS based on revised Bloom’s taxonomy. 3.1 Representations of HOTS and LOTS in the instructional questions in the English textbook When English Rings a Bell grade VIII All of the cognitive domains were covered by the classification of instructional questions based on the revised Bloom's taxonomy in the textbook. They were the levels of remembering, understanding, applying, analyzing, evaluating, and creating. Based on data analysis, it was seen that chapter 1 represented LOTS with the most dominant instructional questions appearing in the remembering and applying categories. Then, chapter V represented LOTS with instructional questions that appeared in balance between LOTS and HOTS. In this chapter, the most frequent occurrence was instructional questions in remembering category for LOTS as also found by Surtikanti et al. (2020) and instructional questions in the evaluating category for HOTS. Next, chapter VI represented LOTS with the most dominant instructional questions appearing in the remembering category. Furthermore, chapter VII also represented LOTS with the most dominant instructional questions appearing in the remembering category. Meanwhile, chapter XII represented instructional questions that appeared in balance between LOTS and HOTS. In this chapter, the most frequently appearing were instructional questions in the remembering category for LOTS and instructional questions in the evaluating and creating categories for HOTS. Chapter XIII, represented HOTS with the most dominant instructional questions appearing in the analyzing and evaluating categories. Hence, the instructional questions in the textbook were still more dominant in LOTS (58%) than HOTS (42%). Erdiana and Panjaitan (2019) showed that the distribution of HOTS and LOTS questions differed significantly but only in reading skill. The same conclusion focusing on learning outcomes was drawn by Sapkota (2022). Similarly, with different foci, Alzu'bi (2014), Assaly and Smadi (2015), and Fitriani and Kirana (2021) also revealed LOTS (69.6%), (61.4%) and (77%) more dominantly respectively. The nearly balanced occurrence between LOTS (51%) and HOTS (49%) was found by Sucipto and Cahyo (2019) but focusing only on reading activities. The partially similar conclusion was also drawn by Dabbagh and Safai (2019). The following are further explanations regarding the representations of HOTS and LOTS in each section. 3.1.1. The representations of HOTS in instructional questions in the textbook Learners must exercise critical thought, analysis, and evaluation when answering HOTS questions. They push students to think imaginatively and apply what they have learned to novel situations. In textbooks, HOTS questions are generally used at the conclusion of a lesson or chapter to assist students in making connections between the content and real-world situations and to promote a deeper level of comprehension and analysis (Anderson et al., 2001). As a result, the HOTS were incorporated into education, both in schools and at the university level (Surtikanti et al., 2020). ISSN 2621-6485 English Language Teaching Educational Journal 244 Vol. 5, No, December 2022, pp. 240-252 Surono et.al (Analysis of LOTS and HOTS of instructional questions……) Table 3.1 The representation of HOTS in the instructional questions Code of Instructional Questions Instructional Questions Page Skill HOTS Basic Verb 13. “The activities I found most difficult were.” 18 Writing C5 Evaluate 23. “The activities I like most were.” 74 Writing C5 Evaluate 45. “I have just learn to.” 84 Writing C5 Evaluate 76. “What I need to do better is / are” 100 Writing C5 Evaluate 84 “Write three different notes for our brother, sister, and parents.” 205 Writing C6 Create 112. “The activities I found most difficult were.” 222 Writing C5 Evaluate Table 3.1 shows that the English textbook “When English Rings A Bell” had instructional questions that represented HOTS in each chapter (more clearly in table 3.3) distributed in different levels, even though the level of occurrence was less dominant than that of LOTS. This could be seen from the existence of instructional questions that had basic skills towards HOTS such as instructions that asked the students to evaluate and create, which are included in the top ranks in the Bloom taxonomy pyramid. (Anderson et al., 2001) The results of the HOTS analysis of instructional questions in Chapter I entitled "It's English time!" showed that there were only four questions out of 14 questions in the textbook and they belonged to the evaluation level (C5). It is clearly understood more generally that in chapter I the representation of HOTS was lower than that of LOTS (Zaiturrahmi et al., 2017; Fitriani & Kirana, 2021), It indicated that the students could state whether they were good or bad about a certain phenomenon or object that they were asked to write down their reflections during the learning process. Furthermore, other data representing HOTS could be found in Chapter V entitled "I'm so happy for you!" which contained materials that offered help for the students to determine what skills they were learning. It is known that at the evaluation level (C5) there were four questions and at the create level (C6) there was only one question out of 11 questions in the textbook. Meanwhile, the results of the HOTS analysis of Chapter VI entitled "Our Busy Streets" showed that there were complete levels of HOTS, namely analysis level (C4) with three questions, evaluation level (C5) with four, and create level (C6) with one out of 23 questions in the chapter. As shown in the table above, one of the C5 levels that most often appeared was demonstrating instructional questions in writing activities in which the students were asked to write down their reflections during the learning process. Chapter VII entitled "My uncle is a zookeeper" showed the results of the HOTS analysis at the analysis level (C4) as many as two questions, the evaluation level (C5) as many as four, and the create level (C6) as many as three questions out of 28 questions in the textbook. It means that in chapter VII the representation of HOTS was lower than LOTS. Then, chapter XII entitled "Don't forget it!" revealed that the HOTS analysis of instructional questions at C4 level appeared once, C5 level six questions and C6 level also six questions out of 26 questions in the chapter. It is clearly seen that in chapter XII there was a balance between HOTS and LOTS representations which was similar to Sucipto and Cahyo’s (2019). What appeared most often in HOTS were the levels of evaluation and create. As can be seen in the table above, one of the C5 levels that was used the most frequently was showing instructional questions in writing assignments (Sapkota, 2022) where students were required to record their learning process reflections and to evaluate tables and pictures. And for C6 level, the most frequently occurring instructional questions were also related to writing activities, in which the students were asked to write notes for their brothers, sisters, and parents. In chapter XIII entitled "We have a lot of history", the results of the HOTS analysis of the instructional questions showed four questions for both C4 level and C5 level out of 11 questions in the chapter. Hence, in chapter XIII the representation of HOTS was lower than LOTS. What appeared most often in the HOTS was the level of analysis and evaluation which were similar to Assaly and Smadi’s (2015). In C4 levels, the instructional questions were correlated with 245 English Language Teaching Educational Journal ISSN 2621-6485 Vol. 5, No, December 2022, pp. 240-252 Surono et.al (Analysis of LOTS and HOTS of instructional questions……) reading skill, in which the students were asked to find important messages in songs and required to have critical thinking (Sutrisno, 2022). They needed to explore the information they had and actively sought information from a variety of sources relating to the topic (Wale & Bogale, 2021). Then, in C5 levels, the instructional questions were devoted to writing activities (Rindawati, et al., 2014), in which the students were asked to write down their reflections during the learning process. The research findings elaborated above were in line with the research by Zaiturrahmi, et al. (2017) which discussed the level of thinking in each question activity. Evaluation can be categorized into HOTS because evaluation requires the ability to think highly to know how far the implementation is carried out in order to achieve the stated goals. Because create is also associated with HOTS, it challenged the students to design a new product by integrating diverse parts in fresh shapes or patterns which also required high-level thinking skills. HOTS could get least significant attention in the textbook for certain skills (Dabbagh & Safai, 2019) but Sukmawijaya et al., (2020) found differently that HOTS in language skills were integrated well. Meanwhile, Rizkiani (2022) showed that the 2013-curriculum based textbook incorporated HOTS excellently and was relevant to language skills. Then, only 19.4% of HOTS incorporation in the textbook was showed by Erdiana and Panjaitan, (2023). Consequently, the EFL textbook's ability to serve as direction toward a learning objective may be diminished by the absence of HOTS (Ariawan et al., 2022). However, there are other things that were not found in abovementioned researches in specific but revealed in this study, though still related to HOTS, LOTS and instructional questions. 3.1.2. The representations of LOTS in instructional questions in the textbook The majority of LOTS questions are factual or knowledge-based and call on the learner to recollect details from the book. LOTS questions are frequently used in textbooks to check that students have understood the material delivered as well as to reinforce it. They usually serve to give students a foundational understanding of the subject at the start of a lesson or chapter. LOTS could be relied on to make decisions that facilitated the use of HOTS, particularly when doing complex tasks (Kamarulzaman et al., 2017). Table 3.2 The representation analysis of LOTS in the instructional questions Code of Instructional Questions Instructional Questions Page Skill LOTS Basic Verb 1. “Listen to our teacher reading the conversation. Repeat after him/her.” 4 Listening C1 Remember 15. “Listen to our teacher reading on the greeting cards. Repeat after him/her..” 66 Listening C1 Remember 26. “Listen to our teacher reading on the presentation. Repeat after him/her. 76 Listening C1 Remember 68. “Here are what Dayu, Lina and Siti do as their routines, according to Udin.” 98 Reading C1 Remember 96. “Listen and repeat after the teacher reading the notices that often see around schools in the book.” 212 Listening C1 Remember 105. “Read the lyrics of the song loudly, clearly, and correctly.” 218 Speaking C1 Remember Table 3.2 shows that the English textbook “When English Rings A Bell” also contained instructional questions that represented LOTS in each chapter. This could be seen from the existence of instructional questions that had basic skills towards LOTS such as instructions that asked the students to remember as the lowest basic skill in the revised Bloom’s taxonomy pyramid. ISSN 2621-6485 English Language Teaching Educational Journal 246 Vol. 5, No, December 2022, pp. 240-252 Surono et.al (Analysis of LOTS and HOTS of instructional questions……) Based on table 3.2 the LOTS for the instructional questions in Chapter I showed five LOTS questions belonging to C1 level, one question belonging to C2 level, and four questions belonging to C3 level in the textbook. It is clear that chapter I showed all LOTS representation. Mostly the LOTS in chapter I often appeared at the level of remembering, meaning that the students could mention definitions, imitate pronunciation, state structures, recite, and repeat (Nazri et al., 2021). It indicated that the students had only to recall and retain newly learned information. This conclusion was also supported by Rustiyani et al. (2021) in their study of “Pathway to English” textbook and Febriyani et al. (2020) in their study of “Bahasa Inggris SMA/MA/SMK/MAK kelas XII Edisi revisi 2018”. In other words, it was in knowledge level (Ariawan et al., 2022). Furthermore, other data representing LOTS could also be found in Chapter V of the textbook which contained materials that offered help for the students to identify what skills they were learning. The table shows that the LOTS for the instructional questions in Chapter V comprised C1 level with four questions and C2 level with two questions out of 11 questions in the textbook. As can be seen in the table, the students were also asked to recall and retain newly learned information. However, it was clearly seen that the instructional questions in chapter V is in balance (Sucipto & Cahyo’s, 2019) between the LOTS and HOTS representation with six and five questions respectively. The same phenomena also happened to other chapters of the textbook with different number of questions and LOTS levels. Chapter VI contained 12 C1 and three C2 instructional questions; Chapter VII contained 18 C1 and one C2; all instructional questions (13) in chapter XII belonged to C1 level; and chapter XIII contained only three C1 questions which were less than the HOTS ones with eight questions distributed in C3 and C5 levels. Most LOTS questions in the textbook still represented the level of remembering. In this aspect, Sucipto and Cahyo’s (2019) study showed differently informing applying as the most dominant. In most cases, the C1 level verbs were represented in activities in which the students were asked to recall and retain newly learned information. As can be seen in table 3.2 above, the C1 level instructional questions were realized in different activities, namely speaking activities (Viana et al., 2022) emphasizing on word stress and pronunciation, reading activities emphasizing on asking the students to read the examples carefully and then list the routines, and listening activities emphasizing also on word stress and pronunciation. The abovementioned research findings indicated remembering as the most dominant in LOTS. Remembering is the ability to recall information from previously acquired recollections or memories, whether they were just acquired or have been collected over a long period of time (Anderson et al., 2001). Hence, the tendency of LOTS questions appearance was clear (Zainil, 2019). It means that LOTS questions were more dominant (Ariawan et al., 2023). This finding ran counter to Wu and Pei's (2018) assertion that HOTS-based questions should ideally be utilized more frequently, particularly for fostering critical thinking as a crucial 21st-century skill (Alghamdi, 2022). However, there were other things that were not found in their study but revealed in this research, namely the comparison of HOTS and LOTS in all instructional questions of all English skills under investigation. Meanwhile, low English proficiency served as a cognitive barrier for students, preventing them from progressing to the higher levels of Bloom's revised taxonomy (Dabbagh & Safai, 2019). Additionally, LOTS was critical in establishing a framework for the implementation of HOTS (Kamarulzaman et al., 2017). 3.1.3. The occurrence of HOTS and LOTS in the activities in English textbooks. It is a good idea to utilize instructional questions in textbooks to get students interested in the topic and to think critically about the ideas they are learning. Textbooks can offer a thorough learning experience that aids students in developing a solid comprehension of the subject matter by using both LOTS and HOTS questions. These inquiries can be used to gauge students' comprehension of the subject matter and to promote deeper learning and critical thinking. Regarding the point, the occurrence of HOTS and LOTS in the textbook can be seen in Table 3.3. 247 English Language Teaching Educational Journal ISSN 2621-6485 Vol. 5, No, December 2022, pp. 240-252 Surono et.al (Analysis of LOTS and HOTS of instructional questions……) Table 3.3 The frequency and Percentage of activities in English textbook Chapter Levels of Cognitive Domain Total C1 C2 C3 C4 C5 C6 I 5 1 4 - 4 - 14 V 4 2 - - 4 1 11 VI 12 3 - 3 4 1 23 VII 18 1 - 2 4 3 28 XII 13 - - 1 6 6 26 XIII 3 - - 4 4 - 11 Total 55 7 4 10 26 11 113 Percentage 49% 6 % 3 % 9 % 23 % 10 % 100 % The research revealed that the activities in the English textbook required both LOTS and HOTS. Partly, this corresponded to the research by Zaiturrahmi, et al. (2017) which discussed that the majority of the listening, reading, speaking, and writing activities placed a strong emphasis on LOTS. Additionally, the authors struggled to significantly modify the cognitive domain of the speaking, listening, reading, and writing activities. It was a wonderful thing that the authors of the textbook included certain materials that possibly inspired and encouraged students to use all of their cognitive capabilities to the possible maximum extent. Although the quantity of instructional questions demanding a high level of cognitive ability did not yet meet the standard for the educational objectives, it was clear that the authors had taken this into account when encouraging the students to use HOTS. The distribution of HOTS in the textbook was not significantly unequal which was in contrast to Febriyani’s et al. (2020) conclusion. A very sharp imbalance between LOTS (97%) and HOTS (3%) was found by Azizah and Bharati (2021) who analyzed Package B textbooks or modules for the students of non-formal equivalency education. According to the Ministry of Education and Culture, students must learn to use HOTS and develop the ability to make realistic assumptions in accordance with the Curriculum 2013 mindset, but it was not commonly promoted through textbooks (Peyró et al., 2020). However, because 58% of the activities in the textbook only focused on LOTS, the authors of the textbook found it challenging to succeed in their goal of inspiring students to be HOTS. They were evidently unaware of the K13 curriculum's emphasis on developing students' HOTS. The studies by Sapkota (2022), Razmjoo and Kazempourfard (2012), Igbaria (2013), and Alzu’bi (2014) also supported this finding. Their research revealed that the authors of English textbooks created materials that placed emphasis more on LOTS than HOTS, despite the curriculum's stated objectives of developing students' critical thinking to be more creative, imaginative, and competent. Nastiti’s (2020) study also showed similar point, but she did not focus her study on HOTS, LOTS and instructional questions specifically. Regarding the elaboration above, the English textbook “When English Rings a Bell” grade VIII was determined to be less relevant to the 2013 Curriculum since it placed emphasis more on LOTS than HOTS in the instructional questions. But the imbalance between LOTS (58%) and HOTS (42%) in the textbook was still tolerable. The explanation above indicated that out of 113 questions in the textbook, remembering skills were realized in 55 questions, understanding skills in seven questions and applying skills in four questions. It implied that out of 113 questions in the textbook, analyzing skills were realized in 10 questions, evaluating skills in 26 questions and creating skills in 11 questions. Finally, most of the instructional questions that were dominantly found in the textbook were in the LOTS category (chapters I, V, VI, VII, and XII). Only one chapter dominated the HOTS category (Chapter XIII). Hence, critical thinking-oriented tasks in educational materials require the establishment of national norms and must begin at the top of the educational system in order to transform and affect the entire curriculum over time (Susandari et al., 2019). As also stated by Aryani and Wahyuni (2020), the current findings appeared crucial for educators, textbook authors, and academicians as they had to collaborate to create HOTS-based textbooks in the future. ISSN 2621-6485 English Language Teaching Educational Journal 248 Vol. 5, No, December 2022, pp. 240-252 Surono et.al (Analysis of LOTS and HOTS of instructional questions……) 4. Conclusion Considering the discussion above, it can be concluded that the textbook "When English Rings A Bell" for grade VIII 2017 Revised Edition contained the instructional questions comprising HOTS and LOTS. However, they were not represented in balance emphasizing the LOTS levels instead of the HOTS ones. Although they were not balanced, at least the authors have tried to include the HOTS materials. Even, the imbalance was still tolerable. Then, the HOTS were also represented quite well in this textbook. There was only one out of 13 chapters which represented the HOTS in the textbook. It implies that the textbook will continue stressing the LOTS which can be more easily internalized by the students in learning the textbook materials. However, to improve students' critical thinking, it is suggested that the textbook authors balance the level of thinking in instructional questions proportionally and teachers intelligently employ more HOTS than LOTS creatively. Acknowledgment The researcher extends the gratitude to all parties who helped in the process of this research project, data collection, data analysis and provided constructive feedback so that this paper could be completed. Therefore, the researchers would like to thank and appreciate to all parties who have supported and encouraged the researchers to complete this manuscript. Declarations Author contribution : Surono was responsible for the entire research project. He also led the writing of the manuscript and the collaboration with the second author. Bambang W. Pratolo revised and proofread the final draft. Siti LH. participated in the data collection, transcription and analysis. All the authors approved the final manuscript. Funding statement : This research did not receive any funding from any party. Conflict of interest : All authors declare that they have no competing interests. Ethics declaration Additional information : : We as authors acknowledge that this work has been written based on ethical research that conforms with the regulations of our university and that we have obtained the permission from the relevant institute when collecting data. We support English Language Teaching Educational Journal (ELTEJ) in maintaining high standards of personal conduct, practicing honesty in all our professional practices and endeavors. No additional information is available for this paper. Conflict of interest : The authors declare no conflict of interest. Additional information : No additional information is available for this paper. REFERENCES Alghamdi, A (2022). Empowering EFL students with 21st-century skills at a Saudi University: Challenges and opportunities. International Journal of Education, Learning, and Development, 10(3), 39-53. https://doi.org/10.37745/ijeld.2013/vol10no3pp.39-53 Alzu’bi, M. A. (2014). The Extend of Adaptation Bloom’s Taxonomy of Cognitive Domain In English Questions Included in General Secondary Exams. Advances in Language and Literary Studies, 5(2), 67–72. https://doi.org/10.7575/aiac.alls.v.5n.2p.67 Anderson, L. W., Krathwohl, P. W., Airasian, D. R., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., Raths, J., & Wittrock, M. C. (2001). Taxonomy for_ Assessing a Revision 0f Bloom’s 249 English Language Teaching Educational Journal ISSN 2621-6485 Vol. 5, No, December 2022, pp. 240-252 Surono et.al (Analysis of LOTS and HOTS of instructional questions……) Taxonomy of Educational Objectives. https://www.uky.edu/~rsand1/china2018/texts/Anderson-Krathwohl - A taxonomy for learning teaching and assessing.pdf Apichat, B. & Fatimah, N. (2022). Students’ difficulties in learning English speaking: A case study in a Muslim high school in the South of Thailand. Teaching English as a Foreign Language Journal, 1(1), 13-22. http://journal1.uad.ac.id/index.php/tefl/index Ariawan, S., Kholidi, M.A., & Putra, M., (2023). The level of thinking skills in the reading section of EFL textbook in Indonesia. VELES: Voices of English Language Education Society, 7(1). 117-125. http://e-journal.hamzanwadi.ac.id/index.php/veles/index Arifin, S., Ilyas, H. F., & Sukmawidjaya, M. (2020). Using journal entries and assigned writing to promote students' critical thinking. VELES: Voices of English Language Education Society, 4(1), 106–117. https://doi.org/10.29408/veles Aryani, E. J., & Wahyuni, S. (2020). An analysis of higher order thinking skills realization in reading comprehension questions. Language Circle: Journal of Language and Literature, 15(1), 83–89. DOI: https://doi.org/10.15294/lc.v15i1.26064 Assaly, I.R., & Smadi, O.M. (2015). Using Bloom’s Taxonomy to evaluate the cognitive levels of Master Class textbook’s questions. English Language Teaching, 8(5), 100 – 110. https://doi.org/10.5539/elt.v8n5p100 Azizah, F. U., & Bharati, D. A. L. (2021). HOTS in Reading Activities of Modules 1 to 5 Easy English for Package B. English Education Journal (EEJ), 11(4). 600-607. http://journal.unnes.ac.id/sju/index.php/eej Changwong, K., Sukkamart, A., & Sisan, B. (2018). Critical thinking skill development: Analysis of a new learning management model for Thai high schools. Journal of International Studies. https://doi.org/10.14254/2071- 8330.2018/11-2/3 Ching, H.Y., & da Silva, E.C. (2017). The Use of Bloom’s Taxonomy to develop competences in students of a business undergraduate course. Journal of International Business Education, 12(1): 107-126. https://doi.org/10.5465/AMBPP.2017.10153 Dabbagh, A., & Safai, A., (2019). Comparative Textbook Evaluation: Representation of Learning Objectives in Locally and Internationally Published ELT Textbooks. Issues in Language Teaching (ILT), 8(1). 249-277. DOI: 10.22054/ilt.2020.48071.440 Elo, S., Kääriäinen, M., Kanste, O., Pölkki, T., Utriainen, K., & Kyngäs, H. (2014). Qualitative Content Analysis. Sage Open, 4(1), 215824401452263. https://doi.org/10.1177/2158244014522633 Erdiana, N., & Panjaitan, S. (2023). How is HOTS integrated into the Indonesian high school English textbook? Studies in English Language and Education, 10(1), 60-77. https://doi.org/10.24815/siele.v10i1.26052 Errington, A., & Bubna-Litic, D. (2015). Management by Textbook: The Role of Textbooks in Developing Critical Thinking. Journal of Management Education, 39(6), 774–800. https://doi.org/10.1177/1052562915594839 Fahmi, I. N. (2020). Higher order thinking skill questions in reading comprehension exercise of when English rings a bell textbook. Retain, 8(4), 187–193. https://jurnalmahasiswa.unesa.ac.id/index.php/43/article/view/35789 Febrina, U., Usman, B., & Muslem, A. (2019). Analysis of reading comprehension questions by using revised Bloom’s taxonomy on higher order thinking skill (HOTS). English Education Journal, 10(1), 1-15. https://jurnal.usk.ac.id/EEJ/article/view/13253 ISSN 2621-6485 English Language Teaching Educational Journal 250 Vol. 5, No, December 2022, pp. 240-252 Surono et.al (Analysis of LOTS and HOTS of instructional questions……) Febriyani, R. A., Yunita, W., & Damayanti, I. (2020). An analysis on higher order thinking skill (HOTS) in compulsory English textbook for the twelfth grade of Indonesian senior high schools. Journal of English Education and Teaching (JEET), 4(2). 170-183. https://doi.org/10.33369/jeet.4.2.170-183 Fitriani, D.H., & Kirana, D.P. (2021). Higher order thinking skill (HOTS) in English language textbook in senior high school. International Journal of Research on English Teaching and Applied Linguistics, 2(2), 55 – 66. DOI : 10.30863/ijretal.v2i2.2455 Gilakjani, A. P., & Sabouri, N. B. (2016). Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review. English Language Teaching, 9(6), 123. https://doi.org/10.5539/elt.v9n6p123 Hafizhah, R. & Pratolo, B. W. (2022). The effectiveness of using crossword puzzle games to improve vocabulary mastery of the 7th grade students of SMP Muhammadiyah 2 Kalasan. Teaching English as a Foreign Language Journal, 1(1) 1-12. http://journal1.uad.ac.id/index.php/tefl/index Hasanah, H. (2017). Teknik-Teknik Observasi (Sebuah Alternatif Metode Pengumpulan Data Kualitatif Ilmu-ilmu Sosial). At-Taqaddum, 8(1), 21-46. https://doi.org/10.21580/at.v8i1.1163 Hatmanto, E., D., Pratolo, B. W., & Baskoro, C. (2023). Unveiling the digital classroom: Exploring students' perspectives on engaging online discussions in English language education at a private University in Yogyakarta. Teaching English as a Foreign Language Journal, 2(1). 10-19. https://doi.org/10.12928/tefl.v2i1.429 Igbaria, A. K. (2013). A content analysis of the WH-questions in the EFL textbook of horizons. International Education Studies, 6(7), 200–224. https://doi.org/10.5539/ies.v6n7p200 Irianti, L., Febriani, R. B., & Friatin, L. Y. (2022). Promoting students' higher order thinking through flipped classroom model in listening comprehension classes. VELES: Voices of English Language Education Society, 6(1), 201–214. https://doi.org/10.29408/veles.v6i1.5060 Kamarulzaman, M. S., Sailin, S. N., Mahmor, N.A., & Shaari, A.J. (2019). Correlation between LOTS and HOTS scores among UUM students. Asian Journal of Educational Research, 5(3). 71-76. https:/www.multidisciplinaryjournals.com Kelly, M. (2014). Bloom's Taxonomy in the Classroom. [Online] Available in http://712educators.about.com/od/testconstruction/p/bloomstaxonomy.htm Lestari, D. D., Surono, & Ika, S. (2022). Cultural contents of senior high school English textbook entitled “Bahasa Inggris”. Teaching English as a Foreign Language Journal, 1(2) 103-113. http://journal1.uad.ac.id/index.php/tefl/index Miles, M. B., Huberman, A. M., & Saldana, J. (2014). Qualitative data anlaysis: A methods sourcebook. 3rd ed. SAGE Publications, Ltd. Nastiti, A. A. (2020). An Analysis of English Textbook Focusing On Reading Material To Enhance Students’ Critical Thinking in Recount Text and Narrative Text. Retain, 69–79. https://jurnalmahasiswa.unesa.ac.id/index.php/43/article/view/33967 Nazri, M. A., Nursaly, M. A. R., Murcahyanto, H., & Ernawati, T. (2021). Bahasa Inggris Sebagai Pendidikan Karakter pada Full Day School di Madrasah Aliyah. JOEAI: Journal of Education and Instruction, 4(2), 457–463. https://doi.org/https://doi.org/10.31539/joeai.v4i2.2822 Peyró, M. C. R., Herrero, E. C., & Pérez, E. L., (2020). Thinking skills in Primary Education: An Analysis of CLIL Textbooks in Spain. Porta Linguarum, 33(33). 183-200. DOI: 10.30827/portalin.vi33.26647 Purwanti, E., & Anggraini, R., Agustina, N. Q., (2023). Teacher professional development during COVID-19 pandemic: Opportunities and challenges. Teaching English as a Foreign Language Journal, 2(1) 1- 9. https://doi.org/10.12928/tefl.v2i1.426 Qasrawi, R., & BeniAndelrahman, A. (2020). The higher and lower-order thinking skills (HOTS and LOTS) in Unlock English textbooks (1st and 2nd editions) based on Bloom’s Taxonomy: An 251 English Language Teaching Educational Journal ISSN 2621-6485 Vol. 5, No, December 2022, pp. 240-252 Surono et.al (Analysis of LOTS and HOTS of instructional questions……) analysis study. International Online Journal of Education and Teaching (IOJET), 7(3). 744-758. https://iojet.org/index.php/IOJET/article/view/866 Razmjoo, S. A., & Kazempourfard, E. (2012). On the representation of Bloom’s revised taxonomy in interchange coursebooks. The Journal of Teaching Language Skills, 4(1), 171–204. https://doi.org/10.22099/JTLS.2012.336 Rindawati, Ikhsanudin, & Wardah. (2014). An Analysis on English Textbook “Bahasa Inggris: When English Rings The Bell.” Jurnal Pendidikan Dan Pembelajaran Khatulistiwa, 3(9), 1-13 https://jurnal.untan.ac.id/index.php/jpdpb/article/view/6871/7082 Rizkiani, F. (2022). The integration of hots in the lesson objectives of English textbook for second graders. BAHTERA: Jurnal Pendidikan Bahasa dan Sastra, 21(2), 189 – 201. http://journal.unj.ac.id/unj/index.php/bahtera Rustiyani, Sofyan, D., & Syafryadin. (2021). Levels of cognitive domain of tasks in English textbooks for senior high school: A revised Bloom’s taxonomy analyses. English Education: Jurnal Tadris Bahasa Inggris, 14(2). 92-105. https://ejournal.radenintan.ac.id/index.php/ENGEDU Sapkota, A. (2022). Relevancy of revised Bloom’s taxonomy in school-level English language curriculum. NELTA Bagmati Journal. 3(1). 19 – 40. DOI: https://doi.org/10.3126/nbj.v3i1.53413 Schilling, J. (2006). On the pragmatics of qualitative assessment: Designing the process for content analysis. European Journal of Psychological Assessment, 22(1), 28- 37. https://doi.org/10.1027/1015-5759.22.1.28 Soviyah (2022). Investigating the motivational teaching strategies of an English native speaker lecturer. Teaching English as a Foreign Language Journal, 1(1), 30-41. http://journal1.uad.ac.id/index.php/tefl/index St. Pierre, E.A., & Jackson, A. Y. (2014). Qualitative data analysis after coding. Qualitative Inquiry, 20(6). 715-719. https://doi.org/10.1177/1077800414532435 Sucipto, S., & Cahyo, S. D. (2019). A Content Analysis of the Reading Activities in “Bright 2”an English Textbook for Junior High School Students. English Language Teaching Educational Journal, 2(1), 13. https://doi.org/10.12928/eltej.v2i1.918 Sukmawijaya, A., Yunita, W., & Sofyan, D. (2020). Analysing higher order thinking skills on the compulsory English textbook for tenth graders of Indonesian senior high schools. JOALL (Journal of Applied Linguistics and Literature), 5(2). 137-148. Doi: http://dx.doi.org/10.33369/joall.v5i2.10565 Surtikanti, M.W., Mustika Aji Hertanto, M. A., & Agung, A. S. S. N. (2020). Developing HOTS- Based Critical Reading Textbook for Junior High Schools in Ngabang. Metathesis: Journal of English language literature and teaching, 4(2). 205-2016. DOI: 10.31002/metathesis.v4i2.2784 Susandari, S., Warsono, W., & Faridi, A. (2019). The evaluation of exercises compatibility between revised Bloom’s taxonomy and 2013 curriculum reflected in English textbook. English Education Journal, 10(1), 252-265. DOI 10.15294/EEJ.V10I1.35742 Sutama, S., Fuadi, D., Narimo, S., Hafida, S. H. N., Novitasari, M., Anif, S., Prayitno, H. J., Sunanih, S., & Adnan, M. (2022). Collaborative mathematics learning management: Critical thinking skills in problem solving. International Journal of Evaluation and Research in Education (IJERE), 11(3), https://doi.org/10.11591/ijere.v11i3.22193 Sutrisno, D. (2022). Fostering student's critical reading through technology integrated instruction. Teaching English as a Foreign Language Journal, 1(1), 125-134. http://journal1.uad.ac.id/index.php/tefl/index Tomlinson, B., & Masuhara, H. (2018). The Complete Guide to Theory and Practice of Materials Development for Language Learning. Wiley. https://www.ebsco.com/terms-of-use ISSN 2621-6485 English Language Teaching Educational Journal 252 Vol. 5, No, December 2022, pp. 240-252 Surono et.al (Analysis of LOTS and HOTS of instructional questions……) Viana, S. A., Suhandra, R. I., & Ariawan, S. (2022). The Analysis of EFL Students' Level of Thinking Skill in Speaking Activities. 1, 103–109. Tadris Bahasa Inggris IAIN Kediri: https://faktarbiyah.iainkediri.ac.id/2022/08/24/1st-international-conference-on-english- language-teaching/ Wale, B. D., & Bogale, Y. N. (2021). Using inquiry-based writing instruction to develop students’ academic writing skills. Asian-Pacific Journal of Second and Foreign Language Education, 6(1). https://doi.org/10.1186/s40862-020-00108-9 Wu, Y., & Pei, Z. (2018). An investigation of critical thinking manifested in the questions of EFL textbooks for tertiary-level English majors of China. American Journal of Education and Learning, 3(2), 72-84. https://doi.org/10.20448/804.3.2.72.84 Zainil, Y., Rosa, R. N. & Fitrawati. (2019). An analysis of reading comprehension questions in English textbooks for SMAN Padang: HOTS. Proceedings of the Eight International Conferences on Languages and Arts (ICLA_2019), 76-80 http://creativecommons.org/licenses/by-nc/4.0/). Zaiturrahmi, Kasim, U., & Zulfikar, T. (2017). Analysis of instructional questions in an English textbook for senior high schools. English Education Journal (EEJ), 8(4), 536–552. https://jurnal.unsyiah.ac.id/EEJ/article/view/9217/7224