
































Microsoft Word - Surono 240-252.docx


English Language Teaching Educational Journal       ISSN 2621-6485 
Vol. 5, No. 3, December 2022, pp. 240-252  

        https://doi.org/10.12928/eltej.v5i3.8168           http://journal2.uad.ac.id/index.php/eltej/index        eltej@pbi.uad.ac.id 

Analysis of HOTS and LOTS of instructional questions in the 
English textbook "When English Rings a Bell" for grade VIII  
Surono a,1*, Bambang Widi Pratolob,2, Siti Latifah Hanun c,3 
a, b, c Universitas Ahmad Dahlan, Jl. Ringroad Selatan, Kragilan, Tamanan, Kec. Banguntapan, Kabupaten Bantul  Daerah Istimewa Yogyakarta 55191 
1 surono@pbi.uad.ac.id, 2bambang.pratolo@pbi.uad.ac.id, 3sitilatifahhanun669@gmail.com   
* Corresponding author 

 

ARTI CLE  I NFO 
 

ABSTRACT   
 

 
Article history 
Received 13 September 2022 
Revised 25 October 2022 
Accepted 5 December 2022 
 

English textbooks have an essential role in learning English subjects for 
students in junior high school, so the quality of textbooks must be of high 
quality. To make students master English language skills, they need a 
good quality of textbooks to improve their critical thinking skills. This 
research aims to find out the representation of higher order thinking skills 
(HOTS) and lower order thinking skills (LOTS) and the appearance 
frequency in the instructional questions in English textbook entitled 
“When English Rings A Bell” grade VIII. This study used a content 
analysis with qualitative approach. Six chapters of the textbook were 
taken as the sample. Observation and note-taking techniques were used 
to collect the data using observation sheet. The data were analyzed by 
using Miles’s et al. model regarding coding method. The research 
findings showed that chapters I, VI, and VII represented LOTS with the 
most dominant in remembering and applying. Meanwhile, the 
appearance of LOTS and HOTS in chapters V and XII looked nearly in 
balance dominating the verb remembering for LOTS and the verbs 
evaluating and creating for HOTS. The realization of HOTS in the 
chapters was less than LOTS, except chapter XIII. After the instructional 
questions in all activities were categorized, the distribution of LOTS in 
the textbook appeared in 66 (58%) and HOTS in 47 questions (42%). It 
is concluded that the textbook distributed less evenly in HOTS and LOTS 
composition. It implies that the textbook still focuses on LOTS. It is 
recommended that the authors of the textbook balance the HOTS and 
LOTS in the instructional questions proportionally to encourage students 
to think critically. Teachers should creatively modify the instructional 
questions to meet the balance. 

 
 
 

 
 

 
This is an open access article under the CC–BY-SA license. 

    

 

 
Keywords 
HOTS 
LOTS 
Instructional Questions 
Content Analysis 
Textbook 
 
 
 
 
 

 

How to Cite: Surono, Pratolo. B. W., & Hanun, L. S. (2022). Analysis of HOTS and LOTS of instructional 
questions in the English textbook “When English Rings a Bell” for grade VIII. English Language Teaching 
Educational Journal, 5 (3), 240-252. https://doi.org/10.12928/eltej.v5i3.8168 

1. Introduction 
Nowadays, education must go beyond simply retaining material and memorization (Qasrawi 

& BenyAbdelrahman, 2020). Higher order thinking skills (HOTS) and lower order thinking skills 
(LOTS) are terms used in education. The Indonesian government popularized HOTS, and the idea 
was used in the country's national test (Ariawan et al., 2023). HOTS are very often incorporated in 
instructional questions of a textbook. The purpose of instructional questions is to get students to 



241  English Language Teaching Educational Journal  ISSN 2621-6485 
  Vol. 5, No, December 2022, pp. 240-252 
 

Surono et.al (Analysis of LOTS and HOTS of instructional questions……) 

reflect on and interact with the subject matter. Instructional questions are frequently included in 
textbooks to aid students in understanding and applying the principles they are learning including the 
culture of the target language (Soviyah, 2022). Depending on how much thought is needed to respond 
to each topic, they can either be classified as HOTS or LOTS. However, higher cognitive levels are 
necessary to increase students' awareness of their own learning processes and teach them how to use 
such capacities to improve their academic performance. (Ching & da Silva, 2017). 

One of the most crucial and integral parts of the language instruction process is the use of 
textbooks (Dabbagh & Safai, 2019). HOTS development resources and exercises are a requirement 
of a good textbook (Erdiana & Panjaitan, 2023). So, it is essential to develop textbooks with adequate 
material in order to fulfill learning objectives (Lestari et al., 2022). Wale and Bogale (2021) stated 
that textbooks and the educational field are intricately related. A textbook is a piece of educational 
material that is used in schools and universities to support a teaching and learning program, based on 
the modern and widely accepted definition. However, the teachers’ expertise is very crucial to 
complete the role of textbook (Apichat & Fatimah). Additionally, teachers must have the ability to 
modify the learning paradigm to fit the circumstances, including in selecting the textbook (Purwanti 
et al., 2023).  

In the teaching and learning process, textbooks can be used as a guide for teachers and 
students, namely as the main reference or as a supplementary material. In learning activities, students 
are not limited to observing what is explained by the teacher. Students also need references to explore 
knowledge so that their abilities and critical thinking can be optimized. According to Febrina et al. 
(2019), reading a suitable textbook with HOTS issues is essential for students to strengthen their 
critical thinking abilities (Hatmanto et al., 2023), which assist students in navigating constraints and 
successfully resolving problems (Sutama et al., 2022). Thus, teachers must intelligently determine 
which textbooks should be used in the learning process. If a teacher can determine a good textbook, 
it will consequently have a big influence in the learning process (Tomlinson & Masuhara, 2018). 

English assignments from textbooks are crucial to learning. Because of this, instructional 
questions give students the chance to hone their abilities and produce effective teaching and learning. 
By practice, language learning exercises are streamlined and completely grasped. Students are 
required to complete instructional questions from textbooks as one of their exercises (Gilakjani & 
Sabouri, 2016). Because "When English Rings A Bell" is a guide book that highlights the potential 
to build information, abilities, and attitudes as a complement and companion for learning English, 
the researchers chose it as the material for the study. This textbook, which is a component of the 
advanced 2013 curriculum, exhorts students to engage in active learning and critical thinking up until 
the production stage. (Errington & Bubna-Litic, 2015). English textbooks, for example, which are 
skill-based, are seen as a rich source of resources and content that may represent the HOTS and 
LOTS, due to the large number of reading passages and writing activities (Qasrawi & 
BenyAbdelrahman, 2020). 

Unfortunately, many textbooks do not represent HOTS proportionally and only focus on 
LOTS (Hasanah, 2017). In fact, students must also be familiar with the HOTS materials they are 
learning. Therefore, the representation of HOTS and LOTS should be proportional in the English 
textbook. This is because incorporating HOTS in textbooks can help students understand critical 
thinking. In turn, this can help teachers to be able to advance their students through Bloom's 
Taxonomy levels (Kelly, 2014). Critical thinking requires creativity, careful analysis, and the ability 
to adapt (Changwong et al., 2018). 

There have been many researches on textbook analysis with different focuses and intensities. 
Sucipto and Cahyo (2019) analyzed the reading assignments in the English textbook "Bright 2". 
Similarly, Azizah and Bharati (2021) explored reading activities in Package B modules. Then, 
Similarly, Febriyani et al. (2019), Nastiti (2020), Fahmi (2020), Aryani and Wahyuni (2020), 
Ariawan et al. (2023), and Erdiana and Panjaitan (2023) analyzed the HOTS focusing on the reading 
materials from English textbook. Zainil et al. (2019) also looked into the HOTS reading 
comprehension problems found in the English textbooks for a senior high school in Padang. 
Rindawati, et al. (2014) analyzed writing skill in English textbooks to improve students' positive 
attitudes and communicative competence. Meanwhile, Zaiturrahmi, et al. (2017) focused their 
analysis on the exercises of English skills with limited varieties in the textbook entitled "When 
English Rings a Bell" for class X. They also revealed HOTS and LOTS in the exercises in ELT 



ISSN 2621-6485 English Language Teaching Educational Journal   242 
  Vol. 5, No, December 2022, pp. 240-252 

Surono et.al (Analysis of LOTS and HOTS of instructional questions……) 

textbooks, and they focused more on discussing textbook quality. However, they only focused on 
different aspects and language skills in the textbook. The same thing happened to HOTS they 
analyzed in instructional questions which tended to only discuss one English skill. Though analyzing 
HOTS, Rizkiani (2022) just focused on the integration of HOTS in lesson objectives for elementary 
school textbook. In more general idea, the idea of fostering critical thinking in speaking was 
discussed (Viana et al., 2022), writing (Arifin et al., 2020), and listening abilities (Irianti et al., 2022). 
A similar study but different in viewpoint was conducted by Dabbagh and Safai (2019). They 
analyzed Iranian nationwide ELT textbooks. Then, Kamarulzaman et al. (2017) compared HOTS 
and LOTS scores gained by the students. In sum, the previous studies explored HOTS and or LOTS 
in English skills separately. However, though in a bit contrast, this study focused on discussing 
HOTS and LOTS in each instructional question contained in all skills altogether in the textbook. 
Based on the description above, two problems were formulated as the main discussion. They were 
(1) How is the representation of HOTS and LOTS in the instructional questions in the English 
textbook “When English Rings a Bell”? and (2) How frequently are HOTS and LOTS emphasized 
in the activities in the English textbook? These two issues are very important to investigate because 
the EFL textbooks published by the Indonesian government are used by English teachers throughout 
Indonesia, and thus are expected to improve the quality of the instructional questions contained in 
the textbooks. 

This study aims to analyze the representation and frequencies of instructional questions 
considering the HOTS and LOTS. In terms of cognitive domains, the revised Bloom’s taxonomy 
(Anderson et al., 2001) indicates that there are two categories of cognitive domains that must be 
included in the textbook, namely HOTS and LOTS. In LOTS, what students do is usually in the form 
of taking notes, copying, memorizing, or following. In other words, LOTS include remembering 
(C1), which refers to the ability to retrieve, recall, or recognize pertinent knowledge from long-term 
memory; students may mention definitions, imitate pronunciations, state structures, pronounce them, 
and repeat; understanding (C2), which requires students to explain an idea, principle, rules, or 
practice in one or more ways to show that they have understood; and applying (C3), that is, students 
can apply their understanding in a new scenario by employing information or a skill in a new situation 

HOTS is defined as the ability to think strategically to use information in solving problems, 
analyzing arguments, negotiating issues, or making predictions. It can be concluded that the 
understanding of HOTS is a complicated level thinking ability that requires the use of description, 
conclusion, analysis, and other higher order thinking skills and actual instances. Anderson et al. 
(2001) states that HOTS include analyzing (C4), namely the skill of being able to break down 
concepts into their component parts and connect them in order to fully understand the concept; 
evaluating (C5) is the ability to determine the degree of something based on certain norms, criteria 
or benchmarks; and creating (C6), namely the ability to integrate pieces into a new whole and broad 
form, or to create something original. Therefore, it is very important to design textbooks with 
appropriate content of LOTS and HOTS so that learning objectives can be achieved. The implication 
of this research is that it can provide valuable information to the stakeholders involved in learning in 
the classroom, curriculum design, and syllabus development. In addition, materials development also 
gets teachers to discover how to better overcome the textbook's weaknesses with innovative methods 
to achieve the learning goals effectively (Hafizhah & Pratolo, 2022). 

2. Method 
This descriptive qualitative study employed content analysis to describe the information 

obtained from analyzing a textbook. The research subject of this research was the English textbook 
"When English Rings A Bell" for grade VIII published by Ministry of Education and Culture of the 
Republic of Indonesia which consisted of 13 chapters with 234 pages in total. The data of this 
research were gathered through observing six chapters selected to represent other chapters and note 
taking on the instructional questions in the English textbook. An observation sheet was used as the 
instrument to collect data which was based on revised Bloom’s taxonomy (Anderson et al., 2001) to 
observe the representation of HOTS and LOTS in the instructional questions. Then, the instructional 
questions and activities contained in each of the six chapters in the textbook were listed. In order to 
obtain valid and reliable data, the researchers conducted a data trustworthiness using four criteria, 
namely credibility, transferability, dependability and confirmability (Elo et al., 2014).  



243  English Language Teaching Educational Journal  ISSN 2621-6485 
  Vol. 5, No, December 2022, pp. 240-252 
 

Surono et.al (Analysis of LOTS and HOTS of instructional questions……) 

To analyze the data, several steps were taken based on Miles’s et al, (2004) approach by 
regarding coding method (St. Pierre & Jackson, 2014; Schilling, 2006). In data reduction, two steps 
were conducted, namely classifying and coding. The classification in this study was based type of 
cognitive domains – HOTS or LOTS. So, to make it easier to classify the HOTS and LOTS in 
textbooks, each instructional question was coded. After that, the first data display step was to make 
an observation sheet assist researchers to find out the number of each type of HOTS and LOTS. After 
the data were calculated, they were input in a table. The data on each page were compiled into a 
chapter, and each chapter was compiled into the entire book. Then, the percentage of the data were 
presented in tables to see the difference in the number between the HOTS and LOTS of instructional 
questions. Finally, after all the analysis processes were carried out and described, the results of the 
analysis were concluded. 

3. Findings and Discussion 
This study set out to find out the representation and frequencies of HOTS and LOTS in 

instructional questions of all the English skills. The following discussion focuses on the analysis of 
the instructional questions from textbook regarding HOTS and LOTS based on revised Bloom’s 
taxonomy. 
3.1 Representations of HOTS and LOTS in the instructional questions in the English 

textbook When English Rings a Bell grade VIII 
All of the cognitive domains were covered by the classification of instructional questions based 

on the revised Bloom's taxonomy in the textbook. They were the levels of remembering, 
understanding, applying, analyzing, evaluating, and creating. Based on data analysis, it was seen 
that chapter 1 represented LOTS with the most dominant instructional questions appearing in the 
remembering and applying categories. Then, chapter V represented LOTS with instructional 
questions that appeared in balance between LOTS and HOTS. In this chapter, the most frequent 
occurrence was instructional questions in remembering category for LOTS as also found by 
Surtikanti et al. (2020) and instructional questions in the evaluating category for HOTS. Next, 
chapter VI represented LOTS with the most dominant instructional questions appearing in the 
remembering category.  

Furthermore, chapter VII also represented LOTS with the most dominant instructional questions 
appearing in the remembering category. Meanwhile, chapter XII represented instructional questions 
that appeared in balance between LOTS and HOTS. In this chapter, the most frequently appearing 
were instructional questions in the remembering category for LOTS and instructional questions in 
the evaluating and creating categories for HOTS. Chapter XIII, represented HOTS with the most 
dominant instructional questions appearing in the analyzing and evaluating categories. Hence, the 
instructional questions in the textbook were still more dominant in LOTS (58%) than HOTS (42%). 
Erdiana and Panjaitan (2019) showed that the distribution of HOTS and LOTS questions differed 
significantly but only in reading skill. The same conclusion focusing on learning outcomes was 
drawn by Sapkota (2022). Similarly, with different foci, Alzu'bi (2014), Assaly and Smadi (2015), 
and Fitriani and Kirana (2021) also revealed LOTS (69.6%), (61.4%) and (77%) more dominantly 
respectively. The nearly balanced occurrence between LOTS (51%) and HOTS (49%) was found by 
Sucipto and Cahyo (2019) but focusing only on reading activities. The partially similar conclusion 
was also drawn by Dabbagh and Safai (2019). The following are further explanations regarding the 
representations of HOTS and LOTS in each section. 
 

3.1.1. The representations of HOTS in instructional questions in the textbook  
Learners must exercise critical thought, analysis, and evaluation when answering HOTS 

questions. They push students to think imaginatively and apply what they have learned to novel 
situations. In textbooks, HOTS questions are generally used at the conclusion of a lesson or chapter 
to assist students in making connections between the content and real-world situations and to promote 
a deeper level of comprehension and analysis (Anderson et al., 2001). As a result, the HOTS were 
incorporated into education, both in schools and at the university level (Surtikanti et al., 2020). 
 



ISSN 2621-6485 English Language Teaching Educational Journal   244 
  Vol. 5, No, December 2022, pp. 240-252 

Surono et.al (Analysis of LOTS and HOTS of instructional questions……) 

Table 3.1 The representation of HOTS in the instructional questions 

Code of 
Instructional 

Questions 
Instructional Questions Page Skill HOTS Basic Verb 

13. “The activities I found most 
difficult were.” 

18 Writing  C5 Evaluate  

23. “The activities I like most were.” 74 Writing  C5 Evaluate  

45. “I have just learn to.” 84 Writing  C5 Evaluate  

76. “What I need to do better is / are” 100 Writing  C5 Evaluate 

84 “Write three different notes for our 
brother, sister, and parents.” 

205 Writing C6 Create 

112. “The activities I found most 
difficult were.” 

222 Writing  C5 Evaluate  

 
Table 3.1 shows that the English textbook “When English Rings A Bell” had instructional 

questions that represented HOTS in each chapter (more clearly in table 3.3) distributed in different 
levels, even though the level of occurrence was less dominant than that of LOTS. This could be seen 
from the existence of instructional questions that had basic skills towards HOTS such as instructions 
that asked the students to evaluate and create, which are included in the top ranks in the Bloom 
taxonomy pyramid. (Anderson et al., 2001) The results of the HOTS analysis of instructional 
questions in Chapter I entitled "It's English time!" showed that there were only four questions out of 
14 questions in the textbook and they belonged to the evaluation level (C5). It is clearly understood 
more generally that in chapter I the representation of HOTS was lower than that of LOTS 
(Zaiturrahmi et al., 2017; Fitriani & Kirana, 2021), It indicated that the students could state whether 
they were good or bad about a certain phenomenon or object that they were asked to write down their 
reflections during the learning process.  

Furthermore, other data representing HOTS could be found in Chapter V entitled "I'm so 
happy for you!" which contained materials that offered help for the students to determine what skills 
they were learning. It is known that at the evaluation level (C5) there were four questions and at the 
create level (C6) there was only one question out of 11 questions in the textbook. Meanwhile, the 
results of the HOTS analysis of Chapter VI entitled "Our Busy Streets" showed that there were 
complete levels of HOTS, namely analysis level (C4) with three questions, evaluation level (C5) 
with four, and create level (C6) with one out of 23 questions in the chapter.  

As shown in the table above, one of the C5 levels that most often appeared was 
demonstrating instructional questions in writing activities in which the students were asked to write 
down their reflections during the learning process. Chapter VII entitled "My uncle is a zookeeper" 
showed the results of the HOTS analysis at the analysis level (C4) as many as two questions, the 
evaluation level (C5) as many as four, and the create level (C6) as many as three questions out of 28 
questions in the textbook. It means that in chapter VII the representation of HOTS was lower than 
LOTS. Then, chapter XII entitled "Don't forget it!" revealed that the HOTS analysis of instructional 
questions at C4 level appeared once, C5 level six questions and C6 level also six questions out of 26 
questions in the chapter. It is clearly seen that in chapter XII there was a balance between HOTS and 
LOTS representations which was similar to Sucipto and Cahyo’s (2019). What appeared most often 
in HOTS were the levels of evaluation and create. As can be seen in the table above, one of the C5 
levels that was used the most frequently was showing instructional questions in writing assignments 
(Sapkota, 2022) where students were required to record their learning process reflections and to 
evaluate tables and pictures. And for C6 level, the most frequently occurring instructional questions 
were also related to writing activities, in which the students were asked to write notes for their 
brothers, sisters, and parents. In chapter XIII entitled "We have a lot of history", the results of the 
HOTS analysis of the instructional questions showed four questions for both C4 level and C5 level 
out of 11 questions in the chapter. Hence, in chapter XIII the representation of HOTS was lower than 
LOTS. What appeared most often in the HOTS was the level of analysis and evaluation which were 
similar to Assaly and Smadi’s (2015). In C4 levels, the instructional questions were correlated with 



245  English Language Teaching Educational Journal  ISSN 2621-6485 
  Vol. 5, No, December 2022, pp. 240-252 
 

Surono et.al (Analysis of LOTS and HOTS of instructional questions……) 

reading skill, in which the students were asked to find important messages in songs and required to 
have critical thinking (Sutrisno, 2022). They needed to explore the information they had and actively 
sought information from a variety of sources relating to the topic (Wale & Bogale, 2021). Then, in 
C5 levels, the instructional questions were devoted to writing activities (Rindawati, et al., 2014), in 
which the students were asked to write down their reflections during the learning process. 

The research findings elaborated above were in line with the research by Zaiturrahmi, et al. 
(2017) which discussed the level of thinking in each question activity. Evaluation can be categorized 
into HOTS because evaluation requires the ability to think highly to know how far the 
implementation is carried out in order to achieve the stated goals. Because create is also associated 
with HOTS, it challenged the students to design a new product by integrating diverse parts in fresh 
shapes or patterns which also required high-level thinking skills. HOTS could get least significant 
attention in the textbook for certain skills (Dabbagh & Safai, 2019) but Sukmawijaya et al., (2020) 
found differently that HOTS in language skills were integrated well. Meanwhile, Rizkiani (2022) 
showed that the 2013-curriculum based textbook incorporated HOTS excellently and was relevant 
to language skills. Then, only 19.4% of HOTS incorporation in the textbook was showed by Erdiana 
and Panjaitan, (2023). Consequently, the EFL textbook's ability to serve as direction toward a 
learning objective may be diminished by the absence of HOTS (Ariawan et al., 2022). However, 
there are other things that were not found in abovementioned researches in specific but revealed in 
this study, though still related to HOTS, LOTS and instructional questions. 
 
3.1.2. The representations of LOTS in instructional questions in the textbook  

The majority of LOTS questions are factual or knowledge-based and call on the learner to 
recollect details from the book. LOTS questions are frequently used in textbooks to check that 
students have understood the material delivered as well as to reinforce it. They usually serve to give 
students a foundational understanding of the subject at the start of a lesson or chapter. LOTS could 
be relied on to make decisions that facilitated the use of HOTS, particularly when doing complex 
tasks (Kamarulzaman et al., 2017). 

Table 3.2 The representation analysis of LOTS in the instructional questions 

Code of 
Instructional 

Questions 
Instructional Questions Page Skill LOTS Basic Verb 

1. 
“Listen to our teacher reading the 
conversation. Repeat after 
him/her.” 

4 Listening  C1 Remember  

15. 
“Listen to our teacher reading on 
the greeting cards. Repeat after 
him/her..” 

66 Listening  C1 Remember  

26. 
“Listen to our teacher reading on 
the presentation. Repeat after 
him/her. 

76 Listening  C1 Remember  

68. 
“Here are what Dayu, Lina and Siti 
do as their routines, according to 
Udin.” 

98 Reading  C1 Remember  

96. 
“Listen and repeat after the teacher 
reading the notices that often see 
around schools in the book.” 

212 Listening  C1 Remember  

105. “Read the lyrics of the song loudly, 
clearly, and correctly.” 

218 Speaking  C1 Remember  

 
Table 3.2 shows that the English textbook “When English Rings A Bell” also contained 

instructional questions that represented LOTS in each chapter. This could be seen from the existence 
of instructional questions that had basic skills towards LOTS such as instructions that asked the 
students to remember as the lowest basic skill in the revised Bloom’s taxonomy pyramid. 



ISSN 2621-6485 English Language Teaching Educational Journal   246 
  Vol. 5, No, December 2022, pp. 240-252 

Surono et.al (Analysis of LOTS and HOTS of instructional questions……) 

Based on table 3.2 the LOTS for the instructional questions in Chapter I showed five LOTS questions 
belonging to C1 level, one question belonging to C2 level, and four questions belonging to C3 level 
in the textbook. It is clear that chapter I showed all LOTS representation. Mostly the LOTS in chapter 
I often appeared at the level of remembering, meaning that the students could mention definitions, 
imitate pronunciation, state structures, recite, and repeat (Nazri et al., 2021). It indicated that the 
students had only to recall and retain newly learned information. This conclusion was also supported 
by Rustiyani et al. (2021) in their study of “Pathway to English” textbook and Febriyani et al. (2020) 
in their study of “Bahasa Inggris SMA/MA/SMK/MAK kelas XII Edisi revisi 2018”.  In other words, 
it was in knowledge level (Ariawan et al., 2022).  

Furthermore, other data representing LOTS could also be found in Chapter V of the textbook 
which contained materials that offered help for the students to identify what skills they were learning. 
The table shows that the LOTS for the instructional questions in Chapter V comprised C1 level with 
four questions and C2 level with two questions out of 11 questions in the textbook. As can be seen 
in the table, the students were also asked to recall and retain newly learned information. However, it 
was clearly seen that the instructional questions in chapter V is in balance (Sucipto & Cahyo’s, 2019) 
between the LOTS and HOTS representation with six and five questions respectively. 

The same phenomena also happened to other chapters of the textbook with different number 
of questions and LOTS levels. Chapter VI contained 12 C1 and three C2 instructional questions; 
Chapter VII contained 18 C1 and one C2; all instructional questions (13) in chapter XII belonged to 
C1 level; and chapter XIII contained only three C1 questions which were less than the HOTS ones 
with eight questions distributed in C3 and C5 levels. Most LOTS questions in the textbook still 
represented the level of remembering. In this aspect, Sucipto and Cahyo’s (2019) study showed 
differently informing applying as the most dominant. In most cases, the C1 level verbs were 
represented in activities in which the students were asked to recall and retain newly learned 
information. As can be seen in table 3.2 above, the C1 level instructional questions were realized in 
different activities, namely speaking activities (Viana et al., 2022) emphasizing on word stress and 
pronunciation, reading activities emphasizing on asking the students to read the examples carefully 
and then list the routines, and listening activities emphasizing also on word stress and pronunciation.  

The abovementioned research findings indicated remembering as the most dominant in 
LOTS. Remembering is the ability to recall information from previously acquired recollections or 
memories, whether they were just acquired or have been collected over a long period of time 
(Anderson et al., 2001). Hence, the tendency of LOTS questions appearance was clear (Zainil, 2019). 
It means that LOTS questions were more dominant (Ariawan et al., 2023). This finding ran counter 
to Wu and Pei's (2018) assertion that HOTS-based questions should ideally be utilized more 
frequently, particularly for fostering critical thinking as a crucial 21st-century skill (Alghamdi, 2022). 
However, there were other things that were not found in their study but revealed in this research, 
namely the comparison of HOTS and LOTS in all instructional questions of all English skills under 
investigation. Meanwhile, low English proficiency served as a cognitive barrier for students, 
preventing them from progressing to the higher levels of Bloom's revised taxonomy (Dabbagh & 
Safai, 2019). Additionally, LOTS was critical in establishing a framework for the implementation of 
HOTS (Kamarulzaman et al., 2017). 

 
3.1.3. The occurrence of HOTS and LOTS in the activities in English textbooks. 

It is a good idea to utilize instructional questions in textbooks to get students interested in 
the topic and to think critically about the ideas they are learning. Textbooks can offer a thorough 
learning experience that aids students in developing a solid comprehension of the subject matter by 
using both LOTS and HOTS questions. These inquiries can be used to gauge students' comprehension 
of the subject matter and to promote deeper learning and critical thinking. Regarding the point, the 
occurrence of HOTS and LOTS in the textbook can be seen in Table 3.3. 

 
 
 
 



247  English Language Teaching Educational Journal  ISSN 2621-6485 
  Vol. 5, No, December 2022, pp. 240-252 
 

Surono et.al (Analysis of LOTS and HOTS of instructional questions……) 

Table 3.3 The frequency and Percentage of activities in English textbook 

Chapter Levels of Cognitive Domain Total 
 C1 C2 C3 C4 C5 C6  
I 5 1 4 - 4 - 14 

V 4 2 - - 4 1 11 

VI 12 3 - 3 4 1 23 

VII 18 1 - 2 4 3 28 

XII 13 - - 1 6 6 26 

XIII 3 - - 4 4 - 11 

Total 55 7 4 10 26 11 113 

Percentage 49% 6 % 3 % 9 % 23 % 10 % 100 % 

        
 

The research revealed that the activities in the English textbook required both LOTS and 
HOTS. Partly, this corresponded to the research by Zaiturrahmi, et al. (2017) which discussed that 
the majority of the listening, reading, speaking, and writing activities placed a strong emphasis on 
LOTS. Additionally, the authors struggled to significantly modify the cognitive domain of the 
speaking, listening, reading, and writing activities. It was a wonderful thing that the authors of the 
textbook included certain materials that possibly inspired and encouraged students to use all of their 
cognitive capabilities to the possible maximum extent. Although the quantity of instructional 
questions demanding a high level of cognitive ability did not yet meet the standard for the educational 
objectives, it was clear that the authors had taken this into account when encouraging the students to 
use HOTS. The distribution of HOTS in the textbook was not significantly unequal which was in 
contrast to Febriyani’s et al. (2020) conclusion. A very sharp imbalance between LOTS (97%) and 
HOTS (3%) was found by Azizah and Bharati (2021) who analyzed Package B textbooks or modules 
for the students of non-formal equivalency education. 

According to the Ministry of Education and Culture, students must learn to use HOTS and 
develop the ability to make realistic assumptions in accordance with the Curriculum 2013 mindset, 
but it was not commonly promoted through textbooks (Peyró et al., 2020). However, because 58% 
of the activities in the textbook only focused on LOTS, the authors of the textbook found it 
challenging to succeed in their goal of inspiring students to be HOTS. They were evidently unaware 
of the K13 curriculum's emphasis on developing students' HOTS. The studies by Sapkota (2022), 
Razmjoo and Kazempourfard (2012), Igbaria (2013), and Alzu’bi (2014) also supported this finding. 
Their research revealed that the authors of English textbooks created materials that placed emphasis 
more on LOTS than HOTS, despite the curriculum's stated objectives of developing students' critical 
thinking to be more creative, imaginative, and competent. Nastiti’s (2020) study also showed similar 
point, but she did not focus her study on HOTS, LOTS and instructional questions specifically. 

Regarding the elaboration above, the English textbook “When English Rings a Bell” grade 
VIII was determined to be less relevant to the 2013 Curriculum since it placed emphasis more on 
LOTS than HOTS in the instructional questions. But the imbalance between LOTS (58%) and HOTS 
(42%) in the textbook was still tolerable. The explanation above indicated that out of 113 questions 
in the textbook, remembering skills were realized in 55 questions, understanding skills in seven 
questions and applying skills in four questions. It implied that out of 113 questions in the textbook, 
analyzing skills were realized in 10 questions, evaluating skills in 26 questions and creating skills in 
11 questions. Finally, most of the instructional questions that were dominantly found in the textbook 
were in the LOTS category (chapters I, V, VI, VII, and XII). Only one chapter dominated the HOTS 
category (Chapter XIII). Hence, critical thinking-oriented tasks in educational materials require the 
establishment of national norms and must begin at the top of the educational system in order to 
transform and affect the entire curriculum over time (Susandari et al., 2019). As also stated by Aryani 
and Wahyuni (2020), the current findings appeared crucial for educators, textbook authors, and 
academicians as they had to collaborate to create HOTS-based textbooks in the future. 



ISSN 2621-6485 English Language Teaching Educational Journal   248 
  Vol. 5, No, December 2022, pp. 240-252 

Surono et.al (Analysis of LOTS and HOTS of instructional questions……) 

4. Conclusion 
Considering the discussion above, it can be concluded that the textbook "When English 

Rings A Bell" for grade VIII 2017 Revised Edition contained the instructional questions comprising 
HOTS and LOTS. However, they were not represented in balance emphasizing the LOTS levels 
instead of the HOTS ones. Although they were not balanced, at least the authors have tried to include 
the HOTS materials. Even, the imbalance was still tolerable. Then, the HOTS were also represented 
quite well in this textbook. There was only one out of 13 chapters which represented the HOTS in 
the textbook. It implies that the textbook will continue stressing the LOTS which can be more easily 
internalized by the students in learning the textbook materials. However, to improve students' critical 
thinking, it is suggested that the textbook authors balance the level of thinking in instructional 
questions proportionally and teachers intelligently employ more HOTS than LOTS creatively. 

Acknowledgment 
The researcher extends the gratitude to all parties who helped in the process of this research 

project, data collection, data analysis and provided constructive feedback so that this paper could be 
completed. Therefore, the researchers would like to thank and appreciate to all parties who have 
supported and encouraged the researchers to complete this manuscript. 

Declarations 
Author contribution : Surono was responsible for the entire research project. He also led the 

writing of the manuscript and the collaboration with the second 
author. Bambang W. Pratolo revised and proofread the final draft. Siti 
LH. participated in the data collection, transcription and analysis. All 
the authors approved the final manuscript.  

Funding statement : This research did not receive any funding from any party. 

Conflict of interest : All authors declare that they have no competing interests. 

Ethics declaration 
 
 
 
 
 
Additional 
information 

: 
 

 

 

 

: 

We as authors acknowledge that this work has been written based on 
ethical research that conforms with the regulations of our university 
and that we have obtained the permission from the relevant institute 
when collecting data. 
  
We support English Language Teaching Educational Journal (ELTEJ) 
in maintaining high standards of personal conduct, practicing honesty 
in all our professional practices and endeavors. 
 
No additional information is available for this paper. 
 

Conflict of interest : The authors declare no conflict of interest. 
Additional 
information 

: No additional information is available for this paper. 
 

REFERENCES 
Alghamdi, A (2022). Empowering EFL students with 21st-century skills at a Saudi University: 

Challenges and opportunities. International Journal of Education, Learning, and Development, 
10(3), 39-53. https://doi.org/10.37745/ijeld.2013/vol10no3pp.39-53 

Alzu’bi, M. A. (2014). The Extend of Adaptation Bloom’s Taxonomy of Cognitive Domain In 
English Questions Included in General Secondary Exams. Advances in Language and Literary 
Studies, 5(2), 67–72. https://doi.org/10.7575/aiac.alls.v.5n.2p.67  

Anderson, L. W., Krathwohl, P. W., Airasian, D. R., Cruikshank, K. A., Mayer, R. E., Pintrich, P. 
R., Raths, J., & Wittrock, M. C. (2001). Taxonomy for_ Assessing a Revision 0f Bloom’s 



249  English Language Teaching Educational Journal  ISSN 2621-6485 
  Vol. 5, No, December 2022, pp. 240-252 
 

Surono et.al (Analysis of LOTS and HOTS of instructional questions……) 

Taxonomy of Educational Objectives. 
https://www.uky.edu/~rsand1/china2018/texts/Anderson-Krathwohl - A taxonomy for learning 
teaching and assessing.pdf 

Apichat, B. & Fatimah, N. (2022). Students’ difficulties in learning English speaking: A case study 
in a Muslim high school in the South of Thailand. Teaching English as a Foreign Language 
Journal, 1(1), 13-22. http://journal1.uad.ac.id/index.php/tefl/index  

Ariawan, S., Kholidi, M.A., & Putra, M., (2023). The level of thinking skills in the reading section 
of EFL textbook in Indonesia. VELES: Voices of English Language Education Society, 7(1). 
117-125. http://e-journal.hamzanwadi.ac.id/index.php/veles/index 

Arifin, S., Ilyas, H. F., & Sukmawidjaya, M. (2020). Using journal entries and assigned writing to 
promote students' critical thinking. VELES: Voices of English Language Education Society, 
4(1), 106–117. https://doi.org/10.29408/veles 

Aryani, E. J., & Wahyuni, S. (2020). An analysis of higher order thinking skills realization in reading 
comprehension questions. Language Circle: Journal of Language and Literature, 15(1), 83–89. 
DOI: https://doi.org/10.15294/lc.v15i1.26064 

Assaly, I.R., & Smadi, O.M. (2015). Using Bloom’s Taxonomy to evaluate the cognitive levels of 
Master Class textbook’s questions. English Language Teaching, 8(5), 100 – 110. 
https://doi.org/10.5539/elt.v8n5p100 

Azizah, F. U., & Bharati, D. A. L. (2021). HOTS in Reading Activities of Modules 1 to 5 Easy 
English for Package B. English Education Journal (EEJ), 11(4). 600-607. 
http://journal.unnes.ac.id/sju/index.php/eej 

Changwong, K., Sukkamart, A., & Sisan, B. (2018). Critical thinking skill development: Analysis of 
a new learning management model for Thai high schools. Journal of International Studies. 
https://doi.org/10.14254/2071- 8330.2018/11-2/3 

Ching, H.Y., & da Silva, E.C. (2017). The Use of Bloom’s Taxonomy to develop competences in 
students of a business undergraduate course. Journal of International Business Education, 12(1): 
107-126. https://doi.org/10.5465/AMBPP.2017.10153 

Dabbagh, A., & Safai, A., (2019). Comparative Textbook Evaluation: Representation of Learning 
Objectives in Locally and Internationally Published ELT Textbooks. Issues in Language 
Teaching (ILT), 8(1). 249-277. DOI: 10.22054/ilt.2020.48071.440 

Elo, S., Kääriäinen, M., Kanste, O., Pölkki, T., Utriainen, K., & Kyngäs, H. (2014). Qualitative 
Content Analysis. Sage Open, 4(1), 215824401452263. 
https://doi.org/10.1177/2158244014522633  

Erdiana, N., & Panjaitan, S. (2023). How is HOTS integrated into the Indonesian high school English 
textbook? Studies in English Language and Education, 10(1), 60-77. 
https://doi.org/10.24815/siele.v10i1.26052 

Errington, A., & Bubna-Litic, D. (2015). Management by Textbook: The Role of Textbooks in 
Developing Critical Thinking. Journal of Management Education, 39(6), 774–800. 
https://doi.org/10.1177/1052562915594839  

Fahmi, I. N. (2020). Higher order thinking skill questions in reading comprehension exercise of when 
English rings a bell textbook. Retain, 8(4), 187–193. 
https://jurnalmahasiswa.unesa.ac.id/index.php/43/article/view/35789  

Febrina, U., Usman, B., & Muslem, A. (2019). Analysis of reading comprehension questions by 
using revised Bloom’s taxonomy on higher order thinking skill (HOTS). English Education 
Journal, 10(1), 1-15. https://jurnal.usk.ac.id/EEJ/article/view/13253 

 
 
 



ISSN 2621-6485 English Language Teaching Educational Journal   250 
  Vol. 5, No, December 2022, pp. 240-252 

Surono et.al (Analysis of LOTS and HOTS of instructional questions……) 

Febriyani, R. A., Yunita, W., & Damayanti, I. (2020). An analysis on higher order thinking skill 
(HOTS) in compulsory English textbook for the twelfth grade of Indonesian senior high schools. 
Journal of English Education and Teaching (JEET), 4(2). 170-183. 
https://doi.org/10.33369/jeet.4.2.170-183 

Fitriani, D.H., & Kirana, D.P. (2021). Higher order thinking skill (HOTS) in English language 
textbook in senior high school. International Journal of Research on English Teaching and 
Applied Linguistics, 2(2), 55 – 66. DOI : 10.30863/ijretal.v2i2.2455 

Gilakjani, A. P., & Sabouri, N. B. (2016). Learners’ Listening Comprehension Difficulties in English 
Language Learning: A Literature Review. English Language Teaching, 9(6), 123. 
https://doi.org/10.5539/elt.v9n6p123 

Hafizhah, R. & Pratolo, B. W. (2022). The effectiveness of using crossword puzzle games to improve 
vocabulary mastery of the 7th grade students of SMP Muhammadiyah 2 Kalasan. Teaching 
English as a Foreign Language Journal, 1(1) 1-12. http://journal1.uad.ac.id/index.php/tefl/index 

Hasanah, H. (2017). Teknik-Teknik Observasi (Sebuah Alternatif Metode Pengumpulan Data 
Kualitatif Ilmu-ilmu Sosial). At-Taqaddum, 8(1), 21-46. https://doi.org/10.21580/at.v8i1.1163 

Hatmanto, E., D., Pratolo, B. W., & Baskoro, C. (2023). Unveiling the digital classroom: Exploring 
students' perspectives on engaging online discussions in English language education at a private 
University in Yogyakarta. Teaching English as a Foreign Language Journal, 2(1). 10-19. 
https://doi.org/10.12928/tefl.v2i1.429 

Igbaria, A. K. (2013). A content analysis of the WH-questions in the EFL textbook of horizons. 
International Education Studies, 6(7), 200–224. https://doi.org/10.5539/ies.v6n7p200 

Irianti, L., Febriani, R. B., & Friatin, L. Y. (2022). Promoting students' higher order thinking through 
flipped classroom model in listening comprehension classes. VELES: Voices of English 
Language Education Society, 6(1), 201–214. https://doi.org/10.29408/veles.v6i1.5060 

Kamarulzaman, M. S., Sailin, S. N., Mahmor, N.A., & Shaari, A.J. (2019). Correlation between 
LOTS and HOTS scores among UUM students. Asian Journal of Educational Research, 5(3). 
71-76. https:/www.multidisciplinaryjournals.com 

Kelly, M. (2014). Bloom's Taxonomy in the Classroom.  [Online] Available in 
http://712educators.about.com/od/testconstruction/p/bloomstaxonomy.htm 

Lestari, D. D., Surono, & Ika, S. (2022). Cultural contents of senior high school English textbook 
entitled “Bahasa Inggris”. Teaching English as a Foreign Language Journal, 1(2) 103-113. 
http://journal1.uad.ac.id/index.php/tefl/index 

Miles, M. B., Huberman, A. M., & Saldana, J. (2014). Qualitative data anlaysis: A methods 
sourcebook. 3rd ed. SAGE Publications, Ltd.  

Nastiti, A. A. (2020). An Analysis of English Textbook Focusing On Reading Material To Enhance 
Students’ Critical Thinking in Recount Text and Narrative Text. Retain, 69–79. 
https://jurnalmahasiswa.unesa.ac.id/index.php/43/article/view/33967 

Nazri, M. A., Nursaly, M. A. R., Murcahyanto, H., & Ernawati, T. (2021). Bahasa Inggris Sebagai 
Pendidikan Karakter pada Full Day School di Madrasah Aliyah. JOEAI: Journal of Education 
and Instruction, 4(2), 457–463. https://doi.org/https://doi.org/10.31539/joeai.v4i2.2822 

Peyró, M. C. R., Herrero, E. C., & Pérez, E. L., (2020). Thinking skills in Primary Education: An 
Analysis of CLIL Textbooks in Spain. Porta Linguarum, 33(33). 183-200. 
DOI: 10.30827/portalin.vi33.26647 

Purwanti, E., & Anggraini, R., Agustina, N. Q., (2023). Teacher professional development during 
COVID-19 pandemic: Opportunities and challenges. Teaching English as a Foreign Language 
Journal, 2(1) 1- 9. https://doi.org/10.12928/tefl.v2i1.426 

Qasrawi, R., & BeniAndelrahman, A. (2020). The higher and lower-order thinking skills (HOTS and 
LOTS) in Unlock English textbooks (1st and 2nd editions) based on Bloom’s Taxonomy: An 



251  English Language Teaching Educational Journal  ISSN 2621-6485 
  Vol. 5, No, December 2022, pp. 240-252 
 

Surono et.al (Analysis of LOTS and HOTS of instructional questions……) 

analysis study. International Online Journal of Education and Teaching (IOJET), 7(3). 744-758. 
https://iojet.org/index.php/IOJET/article/view/866 

Razmjoo, S. A., & Kazempourfard, E. (2012). On the representation of Bloom’s revised taxonomy 
in interchange coursebooks. The Journal of Teaching Language Skills, 4(1), 171–204. 
https://doi.org/10.22099/JTLS.2012.336 

Rindawati, Ikhsanudin, & Wardah. (2014). An Analysis on English Textbook “Bahasa Inggris: When 
English Rings The Bell.” Jurnal Pendidikan Dan Pembelajaran Khatulistiwa, 3(9), 1-13 
https://jurnal.untan.ac.id/index.php/jpdpb/article/view/6871/7082 

Rizkiani, F. (2022). The integration of hots in the lesson objectives of English textbook for second 
graders. BAHTERA: Jurnal Pendidikan Bahasa dan Sastra, 21(2), 189 – 201.  
http://journal.unj.ac.id/unj/index.php/bahtera  

Rustiyani, Sofyan, D., & Syafryadin. (2021). Levels of cognitive domain of tasks in English 
textbooks for senior high school: A revised Bloom’s taxonomy analyses. English Education: 
Jurnal Tadris Bahasa Inggris, 14(2). 92-105. 
https://ejournal.radenintan.ac.id/index.php/ENGEDU 

Sapkota, A. (2022). Relevancy of revised Bloom’s taxonomy in school-level English language 
curriculum. NELTA Bagmati Journal. 3(1). 19 – 40. 
DOI: https://doi.org/10.3126/nbj.v3i1.53413 

Schilling, J. (2006). On the pragmatics of qualitative assessment: Designing the process for content 
analysis. European Journal of Psychological Assessment, 22(1), 28-
37. https://doi.org/10.1027/1015-5759.22.1.28 

Soviyah (2022). Investigating the motivational teaching strategies of an English native speaker 
lecturer. Teaching English as a Foreign Language Journal, 1(1), 30-41. 
http://journal1.uad.ac.id/index.php/tefl/index 

St. Pierre, E.A., & Jackson, A. Y. (2014). Qualitative data analysis after coding. Qualitative Inquiry, 
20(6). 715-719. https://doi.org/10.1177/1077800414532435 

Sucipto, S., & Cahyo, S. D. (2019). A Content Analysis of the Reading Activities in “Bright 2”an 
English Textbook for Junior High School Students. English Language Teaching Educational 
Journal, 2(1), 13. https://doi.org/10.12928/eltej.v2i1.918 

Sukmawijaya, A.,  Yunita, W., & Sofyan, D. (2020). Analysing higher order thinking skills on the 
compulsory English textbook for tenth graders of Indonesian senior high schools. JOALL 
(Journal of Applied Linguistics and Literature), 5(2). 137-148. Doi: 
http://dx.doi.org/10.33369/joall.v5i2.10565  

Surtikanti, M.W., Mustika Aji Hertanto, M. A., & Agung, A. S. S. N. (2020). Developing HOTS-
Based Critical Reading Textbook for Junior High Schools in Ngabang. Metathesis: Journal of 
English language literature and teaching, 4(2). 205-2016. DOI: 10.31002/metathesis.v4i2.2784 

Susandari, S., Warsono, W., & Faridi, A. (2019). The evaluation of exercises compatibility between 
revised Bloom’s taxonomy and 2013 curriculum reflected in English textbook. English 
Education Journal, 10(1), 252-265. DOI 10.15294/EEJ.V10I1.35742 

Sutama, S., Fuadi, D., Narimo, S., Hafida, S. H. N., Novitasari, M., Anif, S., Prayitno, H. J., Sunanih, 
S., & Adnan, M. (2022). Collaborative mathematics learning management: Critical thinking 
skills in problem solving. International Journal of Evaluation and Research in Education 
(IJERE), 11(3), https://doi.org/10.11591/ijere.v11i3.22193  

Sutrisno, D. (2022). Fostering student's critical reading through technology integrated instruction. 
Teaching English as a Foreign Language Journal, 1(1), 125-134. 
http://journal1.uad.ac.id/index.php/tefl/index 

Tomlinson, B., & Masuhara, H. (2018). The Complete Guide to Theory and Practice of Materials 
Development for Language Learning. Wiley. https://www.ebsco.com/terms-of-use 



ISSN 2621-6485 English Language Teaching Educational Journal   252 
  Vol. 5, No, December 2022, pp. 240-252 

Surono et.al (Analysis of LOTS and HOTS of instructional questions……) 

Viana, S. A., Suhandra, R. I., & Ariawan, S. (2022). The Analysis of EFL Students' Level of Thinking 
Skill in Speaking Activities. 1, 103–109. Tadris Bahasa Inggris IAIN Kediri: 
https://faktarbiyah.iainkediri.ac.id/2022/08/24/1st-international-conference-on-english-
language-teaching/ 

Wale, B. D., & Bogale, Y. N. (2021). Using inquiry-based writing instruction to develop students’ 
academic writing skills. Asian-Pacific Journal of Second and Foreign Language Education, 6(1). 
https://doi.org/10.1186/s40862-020-00108-9 

Wu, Y., & Pei, Z. (2018). An investigation of critical thinking manifested in the questions of EFL 
textbooks for tertiary-level English majors of China. American Journal of Education and 
Learning, 3(2), 72-84. https://doi.org/10.20448/804.3.2.72.84 

Zainil, Y., Rosa, R. N. & Fitrawati. (2019). An analysis of reading comprehension questions in 
English textbooks for SMAN Padang: HOTS. Proceedings of the Eight International 
Conferences on Languages and Arts (ICLA_2019), 76-80 
http://creativecommons.org/licenses/by-nc/4.0/). 

Zaiturrahmi, Kasim, U., & Zulfikar, T. (2017). Analysis of instructional questions in an English 
textbook for senior high schools. English Education Journal (EEJ), 8(4), 536–552. 
https://jurnal.unsyiah.ac.id/EEJ/article/view/9217/7224 

 

 

 

 

 

 

 

 

 

 

 

 

 

  


