







































 et al / Journal of English Language Teaching 6 (1) (2017) 

203 

 

 
 

ELT FORUM 9 (2) (2020) 
  

Journal of English Language Teaching 

 
http://journal.unnes.ac.id/sju/index.php/elt 

 

An analysis of the English course book viewed from higher-order thinking skills 

Riski Norita Sari1, Zulfa Sakhiyya2 

English Department, Faculty of Languages and Arts, Universitas Negeri Semarang, Indonesia

 

Article Info 
________________  
Article History: Received in 

14 May 2020  
Approved in 29 November 

2020 

Published in 30 November 

2020 

________________  
Keywords: Higher-Order 
Thinking Skills; Textbook 
Analysis; Variety  
____________________ 

 

 

 

 

Abstract 
___________________________________________________________________ 
This study analyzes an English Course Book entitled Symphony 1 to find out to what 

extent Higher and Lower-Order level thinking used in the reading tasks. The focus of 

this study was on the availability and the variety of HOTS on reading assignments on 
The Symphony 1 English course book. This descriptive qualitative research adopted the 

cognitive domain of Revised Bloom's Taxonomy as the instrument. Subsequently, the 

researcher calculated the percentage and the frequencies of the cognitive field of 

questions analyzed. Afterwards, the items indicated as HOTS are analyzed again by 
using tools from Keshta and Sheif to know the variety of HOTS found in the textbook. 

The results showed that the percentage of LOTS items was 63% while the percentage 
of HOTS items was 37%. It indicated that LOTS dominated the book. This domination 

discourages students from thinking critically. Based on the finding, the researcher 
recommends that textbooks authors need to advance HOTS to develop students' 

critical thinking, and teachers have to improve their knowledge about HOTS. 

 

 
 © 2020 Universitas Negeri Semarang 

  
 Correspondent Address: p-ISSN 2252-6706 | e-ISSN 2721-4532 
B3 Building FBS Unnes  

Sekaran, Gunungpati, Semarang, 50229  

E-mail: riskinorita16@gmail.com  

mailto:riskinorita16@gmail.com


Riski Norita Sari, Zulfa Sakhiyya/ ELT Forum 9 (2) (2020) 

204 

 

 

INTRODUCTION  

Antara (2019) reported Indonesian students have poor literacy; as indicated in PISA (Programme for 

International Student Assessment). Besides, Antara (2019) reported that Makarim, the current Indonesian 

Minister of Education and Culture, stated that Indonesian students’ PISA scores of reading were low in 

2018. Further, the Minister noted that PISA’s report was critical to give perspective for Indonesia Education. 

Moreover, lifelong learning could be built from reading because students would learn almost all materials 

from reading (PISA for Development Reading Framework, 2018). However, the PISA score of reading in 

Indonesia is still low. Also, only 30% of students from Indonesia attained at least level 2 proficiency in 

reading (PISA for Development Reading Framework, 2018). Meanwhile, 77% of students from OECD 

(Organization Co-operation and Development) achieved at least level 2 proficiency in reading. 

Textbooks are widely available in Indonesia. However, most of the textbooks in Indonesia contain 

Lower-Order Thinking Skills because most of the books comprise of memorizing and understanding 

(Margana & Widyantoro, 2017). This shows the reason why it is hard for students to develop their critical 

thinking and reading comprehension. Therefore, a study should be conducted to understand the type of 

order of thinking skills available. Furthermore, Revised Bloom's Taxonomy can ease the researcher to 

differentiate the Higher-Order levels and Lower-Order levels (Anggraeni & Suharyadi, 2013). From the 

findings found, there are 72.5% of Lower-Order thinking skills and 27.5% of Higher-Order thinking skills in 

the textbook. Another research was conducted by Freahat and Smadi (2014). Based on the research, High 

School Stage EFL and New Headway Plus Pre-Intermediate textbooks were dominated by Lower level 

questions. Then the researchers suggest textbooks authors have to balance both questions in Higher-Order 

levels and Lower-Order levels. Similar research conducted by Raqqad and Ismail (2018). This research is 

about analyzing the reading tasks based on Bloom’s Taxonomy. Moreover, Assaly and Igbaria (2014) used 

Bloom’s Taxonomy and the outcome of that study showed that 114 activities presented Lower-Order 

Thinking Skills, while only 59 questions presented Higher-Order Thinking Skills. As a result, Lower-Order 

Thinking Skills dominated the findings. Moreover, Tangsakul et al. (2017) did a research by applying the 

cognitive domain of Revised Bloom’s Taxonomy. The result of the study found that understanding level 

dominated the findings. On the other hand, level cognitive of creating did not exist in the research.  

Textbooks are an essential thing to espouse a learning process because textbooks will guide students 

and teachers in a learning process. Moreover, Abdelrahman (2014) stated that textbooks have instructional 

materials that can support the learning process. Thamrin and Agustin (2019) about Higher-Order Thinking 

Skills. That research led to developing students' critical thinking skills. Besides, they stated that students 

have to acquire the Higher-Order Thinking Skills strategy to improve the thinking process from LOTS to 

HOTS. As a result, students’ critical thinking can be encouraged through HOTS. Thus, teachers should give 

Higher-Order questions related to the material during class activities. Also, Adams (2015) researched 

Bloom's taxonomy of cognitive learning objectives. The study aimed to analyze the relationship between 

Bloom's Taxonomy with critical thinking. The investigation resulted that there is a new dimension across 

all six cognitive processes. Besides, Assaly and Smadi (2015) researched by using Bloom's Taxonomy to 

evaluate the cognitive levels of Master Class Textbook's Questions. This study focuses on assessing the 

cognitive level according to Bloom's Taxonomy. Also, the instrument of this study was used to categorize 

the cognitive level of those questions. The result of the analysis showed that the book analyzed contains 

Lower-Order level and Higher-Order level cognitive. On the other hand, the variation of the cognitive 

domain is not significant. It can be known from the results that 60% is Lower-Order level and 40% is Higher-

Order level. Besides, Anggraeni and Suharyadi (2013) researched reading questions in an English textbook 

based on Bloom’s Taxonomy. Assaly and Smadi (2015) stated textbooks should have sufficient HOTS 

questions to improve students' cognitive level. The ideal distribution of Higher-Order and Lower-Order 

levels is 50:50 (Anggraeni & Suharyadi, 2013). 

Problems investigated in this research are (1) how the availability of HOTS on the reading tasks of 

The Symphony 1 English Course Book is and (2) how the variety of HOTS in the reading tasks of The 

Symphony 1 English Course Book is. Based on the previous studies, some textbooks and tests have been 

analyzed by using Bloom's Taxonomy and or Revised Bloom's Taxonomy. Some of them are also related 

to critical thinking. Although those studies are similar to the present study, however there are differences. 

Most of the studies only focus on the availability of HOTS. Meanwhile, this study also analyzes the variation 

of HOTS analyzed by using the framework from Keshta and Sheif (2013). 

 

 

 



Riski Norita Sari, Zulfa Sakhiyya/ ELT Forum 9 (2) (2020) 

205 

 

 

METHODS 

Qualitative research aims to make new insight. It uses a natural setting to conduct research. According to 

Sherman and Webb (2005), qualitative research has the main goal that is to find new insight from discovery. 

Also, qualitative research is not conducted to conclude the verification of a predetermined idea. 

Subsequently, qualitative research sometimes is called a naturalistic inquiry. In this qualitative research, the 

study aimed to explain HOTS’s composition and variation on the reading tasks of Symphony 1 English 

Course Book. To collect the data, the researcher used content analysis. 

The researcher adopted a qualitative research approach to conduct this study. Qualitative research is 

more iterative, while the content defined by the researcher (Crescentini & Maninardi, 2009). Qualitative 

research aims to make new insight. Qualitative research uses a natural setting to conduct research. 

According to Sherman and Web (2005), qualitative research has the main goal that is to find new insight 

from discovery. Also, qualitative research is not conducted to conclude the verification of a predetermined 

idea. Subsequently, qualitative research sometimes is called as a naturalistic inquiry.  

As a result, definitions and ideas are refined through the process recursively. Besides, qualitative 

research aims to find a holistic picture of the historically odd situation (Ospina, 2004). Content analysis is a 

procedure to categorize messages based on their meaning (Holsti, 1969). In this qualitative research, the 

study aims to explain HOTS's composition and variation on the reading tasks of Symphony 1 English Course 

Book. To collect the data, the researcher used content analysis. Content analysis means the process of 

summarizing and reporting data (Manion, et. al, 2007). Cognitive domain of Revised Bloom's Taxonomy 

was used to analyze the data of this study. Table 1 shows the content analysis using cognitive process of 

Revised Bloom’s Taxonomy for The Symphony 1 English Coursebook. The data acquired are then analyzed 

and divided into remembering, understanding, applying, evaluating, and creating.  

  

  

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The researcher conducted several steps in the procedure of collecting data: 1) reading all the tasks 

inside the textbook, 2) deciding the tasks to be analyzed, and 3) selecting tasks that have the potential to 

support the realization of Higher-Order Thinking Skills. In doing the data analysis, the researcher used the 

method from Milles and Huberman (1994). The data analysis consists of three levels: data reduction, data 

analysis, and conclusion and drawing verification. Then, below is the data reduction conducted by the 

researcher: 1) selecting the data: the researcher read all the tasks inside the book; 2) focusing the data: the 

researcher focused on the questions of reading tasks selected. 3) simplifying the data: the researcher placed 

all questions selected into the table then analyzed by using the instruments; 4) abstracting the data: after the 

data had been  placed, then, the researcher put the data into the classification to know the variety of HOTS; 

and 5) transforming the data. The researcher summed up the Lower-Order and Higher-Order questions of 

the reading tasks from the textbooks in quantitative output to show the percentage and number.  

The data analysis displayed data reduction, after the level of data reduction, then the data analysis 

throughout data display. Data display can extend to the graphic, diagram, chart, or matrix. In the third level 

of the data analysis, the researcher drew the conclusion and verification. The end draws aimed to consider 

what data mean and to assess the implication from the research questions. Meanwhile, confirmation means 

linked integrally to the conclusion drawing, so this aims to verify these new conclusions. 

Table 1.Structure of the cognitive domain and the explanation of the Revised 

Bloom's Taxonomy (Anderson & Krathwohl, 2001) 
Remembering Understanding Applying Analyzing Evaluating Creating 

Recognizing Interpreting Executing Differentiating Checking Generatin

g 

Recalling. Exemplifying Implementing Organizing Critiquing. Planning 

 Classifying  Attributing  Producing 

 Summarizing     

 Inferring     

 Comparing:     

 Explaining.   

 
  



Riski Norita Sari, Zulfa Sakhiyya/ ELT Forum 9 (2) (2020) 

206 

 

 

FINDINGS AND DISCUSSION 

The data for this research was gathered by analyzing the reading tasks from The Symphony 1. Thus, the 

reading tasks from the book analyzed by using the instruments from Anderson and Krathwohl (2001). Then 

questions indicated HOTS were analyzed again by using the tools from Keshta and Sheif (2013). It intended 

to know the variety of HOTS in the reading tasks.  
 

The Availability of HOTS in The Symphony 1 

Some studies previously found that in Indonesia, the textbooks used are more on LOTS (Margana & 

Widyantoro, 2017). It made the researcher wanted to find out the availability of HOTS in the English 

textbook entitled The Symphony 1 English Coursebook. Based on the finding of the book, there are 18 reading 

tasks and 135 reading question items analyzed. The questions were analyzed and classified into six levels of 

the cognitive domain in Revised Bloom's Taxonomy. This table and chart below show the frequency and 

percentage of each level. 

 

Table 2.The cognitive domain of Revised Bloom's 

Taxonomy found in The Symphony 1's reading tasks 

No Categories Frequency 
Percentage 

1 Remembering 52 
39% 

2 Understanding 33 
24% 

3 Applying 0 
0% 

4 Analyzing 43 
32% 

5 Evaluating 6 
4% 

6 Creating 1 
1% 

 

 

  

Figure 1.The cognitive domain of Revised Bloom's Taxonomy found in The Symphony 1's reading tasks 

Based on Table 2, there are six classifications of the question items from the cognitive domain of 

Revised Blooms' Taxonomy. From Figure 1, Lower-Order levels dominated the reading tasks in Symphony 

1 just like a finding in Freahat and Smadi (2014), based on the research High School Stage EFL and New 

Headway Plus Pre-Intermediate textbooks were dominated by Lower Level questions, the most of the 

Remembering
39%

Understanding
24%

Applying
0%

Analyzing
32%

Evaluating
4%

Creating
1%

Remember Understand Apply Analyze Evaluate Create



Riski Norita Sari, Zulfa Sakhiyya/ ELT Forum 9 (2) (2020) 

207 

 

 

coursebook contents were in Lower-Order level, in detail the percentage of Lower-Order level was 63%. 

Like the remembering level was the highest frequency with 50 cases, and the rate was 37%. The 

understanding level recorded 32 cases or 24%. Furthermore, the lower-order level applying did not exist on 

the reading tasks, which was mentioned as 0%.  

Another case in the chart and table was 39% of Higher-Order level questions or 50 questions from the 

total. Afterward, the result shows that the Higher-Order level analyzing level was 32% or 43 items. Besides, 

the Higher-Order level of evaluating level reached 4% or six questions. Moreover, the Higher-Order level 

creating level recorded only 1% or one issue. As a result, creating was the lowest point from the Higher-

Order level, and analyzing was the highest point of the Higher-Order level. The explanation below is the 

detailed information related to the table and graphic. 

 The types of the cognitive process dimension aim to serve a classification for some cognitive processes 

that include in objectives. As shown in the table above, there are some parts of the cognitive process of 

Revised Bloom's Taxonomy, such as remembering, understanding, applying, analyzing, evaluating, and 

creating. Remembering level means by recalling proper knowledge from memories. Besides, intelligence is 

defined as establishing meanings of messages, including oral, written, and graphic communication. 

Moreover, applying mentioned as utilizing knowledge in a given situation. Furthermore, analyzing level 

means splitting materials based on the information's core, after determining how the parts of the report are 

related to each other. Additionally, evaluating means deciding judgment based on a standard. Lastly, 

creating a level, this element is defined by placing some materials from an idea into an original product.  

Along with the data from the analysis, the remembering level used some action verbs such as find, 

name, relate, list, show, and tell. Action verbs dominated by the verb telling with 26 questions. Furthermore, 

there are 14 action verbs of showing, six action verbs of finding, three action verbs of relating, two action 

verbs of listing, and one action verb of naming. Moreover, this analysis also used cognitive processes and 

then found some cognitive processes, such as recalling and recognizing. The number of cognitive process by 

utilizing the recalling process was 46 questions. In contrast, the number of cognitive process by using the 

recognizing method was only six questions. 

The action verbs used on the understanding level were translated, explained, and summarized. There 

were some action verbs found on the book including two translate verbs, 27 describe verbs, and 

3 summarize verbs. As a result, it was dominated by explaining the verb. Also, on the analysis by using the 

cognitive process, there were some processes found, such as interpreting, explaining, and summarizing. In 

detail, the number of interpreting process found was only one. On the other side, the total of explaining 

process reached 29 questions. However, the sum of the summarizing process found were three issues. 

 Along with the data, applying process was not found in the data. Moreover, the one found in the data 

was analyzing level. According to the data, analyzing level used some action verbs such as examine, analyze, 

compare, assume, inspect, classify, discover, inference, distinguish, and compare verb. Analyzing level was 

dominated by examining, and inference verbs. 

 The Evaluating level is one of the Higher-Order levels in Revised Bloom’s Taxonomy. Then, there 

were some action verbs used in the analysis, such as assessing, deciding, and appraising. Action verb 

dominated by assess verb that uses three action verbs. Furthermore, there were two verbs of appraising verb 

and one verb of assess. Besides using action verbs, this analysis also used the cognitive process of critiquing. 

Creating a level is one of the parts of Higher-Order level skills. Then, the analysis of the data resulted in only 

one question included creating a standard. Also, this creating level used to adapt as an action verb. 

Furthermore, the cognitive process found in the data is the generating process. 

Variety of HOTS in The Symphony 1 

Analyzing Level  

Analyzing level is the starting point in Higher-Order levels. According to Nourdad et al. (2018), Higher-

Order Level is the starting stage of the Higher-Order level. Besides, learners can break down the information 

on analyzing level. However, this case also needs calculations and classifications. In the reading tasks, 45 

questions belong to the analyzing level. Cases in the analyzing level are related to breaking up the material 

into smaller part which is engaged with constituent parts and determining how the pieces are associated with 

the whole structure. The cognitive processes in the analyzing level are differentiating, organizing, and 

attributing. Analyzing questions found in the textbook were then analyzed and categorized into 13 criteria, 

as shown in the table below: 

 



Riski Norita Sari, Zulfa Sakhiyya/ ELT Forum 9 (2) (2020) 

208 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

  

 

 

 

 

 

Based on Table 3, there are five out of thirteen cases found in analyzing items of Symphony 1's reading 

tasks. Types included in the analysis are as follows: comparing things in the text, guessing the meaning of 

words in context, reading between the lines, and recognizing clues and evidence. The item using explaining 

the information introduced is the highest number among the five things that are 40% or 18 cases. 

 

Table 4.The example of analyzing item from Symphony 1 
No Items Categories Action 

Verbs 

Cognitive 

Process 

1 What is the meaning of the statement “Pickwick 

was noted for his honesty and humanity”? 

Analyzing inference Attributing 

2 Describe the humor contained in the line “huge 

Pickwick approach him with a whip”. 

Analyzing interpret Differentiating 

  

Table 4. shows the categories of explaining the information introduced. According to Keshta and Seif 

(2013), this category requires students to find the meaning of some statements shown. Then students should 

explain what the purpose of the report is. As a result, students have to understand the comments in the 

context.  

For making students can answer the questions, they can enlighten the statement based on the context. In 

this step of explaining, the information introduced and students should develop their ability to recognize the 

explanation related to the setting. Thus, students should know the meaning of the statement then interpret 

it according to the context. After that, they had to explain in their own words. 

Also, the cognitive process included in the first question was attributing. According to Anderson and 

Krathwohl (2001) attributing is something resulted from a particular thing. The writer defines that the 

attributing process happens from the questions since students should find an explanation about the problem 

context. As a result, students have to find a particular meaning related to the statement. 

Another cognitive process related to this explains the information introduced as differentiating. The 

differentiating cognitive process is to recognize that two things are not the same. This definition means 

students should identify differences between two things or among more than two words. After that, they 

should explain the information about the differences. 

Another item is ‘reading between the lines’. This item means students have to elaborate the text to 

look for the hidden message from the document, which is implicitly stated in the report. Based on the 

analysis, it depicts a percentage of 29% with 13 frequencies. 

 

Table 5.The example of analyzing items from Symphony 1 
No Items Categories Action Verbs Cognitive Process 

1.  What decision does the author make in this poem? Analyzing Examine Attributing 

Table 3.The types of analyzing item in Symphony 1 

No Item Types Frequency Percentage 

1 Distinguishing facts from opinion  0 0 

2 Categorizing information in text  0 0 

3 Comparing items in text  3 7% 

4 Guessing meaning of words in context  1 2% 

5 Reading between lines  13 30% 

6 Recognizing causes and effects  0 0 

7 Explaining the information introduced  18 42% 

8 Recognizing clues and evidence  8 19% 

9 Eliciting rules and principles  0 0 

10 Inferring the mood, attitudes, or tones of the author. 0 0 

11 Distinguishing main ideas from supporting ones. 0 0 

12 Ordering items according to their importance. 0 0 

13 Breaking down the text into its main component. 0 0 

Total 45 100% 



Riski Norita Sari, Zulfa Sakhiyya/ ELT Forum 9 (2) (2020) 

209 

 

 

2.  What do you think the author is saying in this 
poem? 

Analyzing Analyse Attributing 

  

The first question on Table 5 demands students to explore the poem related to the content of the poem. 

To find the answer, students have to know the clues that can support the solution. Usually, the evidence 

appears after the students understand what the meaning of the question is. The second question is almost 

the same as the first question.   

Answering the questions above cannot be found from the lines printed. As a result, the writer 

categorized these questions as reading between the lines items because the answers are not stated implicitly 

in the text, students should find the answer after analyzing the information presented in the book, finding 

the relevant information, and making conclusions related to the questions demanded. Those processes for 

finding the solution also happen in the attributing. In short, those questions are analyzing items. Besides, 

these questions encourage students to make them develop their critical thinking because they should find 

the answer according to the students’ perspective—the students’ perspective which built from the 

information collected.  Another sub-skill found in the analysis is recognizing clues and evidence. In this 

matter, students have to find clues and indications on a particular topic. In this case, below is the example 

for identifying clues and evidence in the text.  

 

Evaluating Level 

The evaluating level includes the Higher-Order Thinking Skills. This cognitive domain level demands 

students to give and defend opinions by presenting judgment sourced from criteria and experience (Anderson 

& Kratwohl, 2001). This category consists of the cognitive process of checking (judgments about internal 

consistency) and critiquing (judgments based on external criteria). In short, this level is higher than the 

analyzing level because, at this level, students urged to make judgments and or opinions after examining the 

text by looking for some point of view. For making a clear explanation, below is the example of evaluating 

level based on the book of The Symphony 1. According to the analysis found in The Symphony 1, this level 

only was counted six-question or 4% from the complete review. Instruments from Keshta & Sheif (2013) 

analyzed the questions. The analysis is presented below: 

 

 Table 6.The types of evaluating items in Symphony 1 

No Item Types Frequency Percentage (%) 

1 Expressing his/her opinion towards situations in the target text  6 100 

2 Concluding themes of texts  0 0 

3 Recognizing subjectivity and objectivity  0 0 

4 Making choices based on reasoned argument  0 0 
Total 6 100 

 

 From those criteria in Table 6, there was only one type found in the textbook. This type encourages 

students to express their arguments based on the situation in the target text. In this level, students should 

make a judgment whether they agree or disagree with the opinion. Then they should find evidence to 

strengthen their argument.  

 

Table 7. The example of evaluating item from Symphony 1 
No Items Categories Action 

Verbs 

Cognitive 

Process 

1 What do you think about Singkawang? Are you 

interested to go there? Give reasons. 

Evaluating Assess Critiquing 

2 Discuss with the class what each local government 

should do to boost income from tourism. 

Evaluating Decide Critiquing 

 

The first question demands students to declare their argument about Singkawang. This evidence-based 

on the situation there. In short, they have to give a reason why they attractive in it. Students have to assess a 

case, so the researcher uses action words to determine. The second question requires students to take action 

toward the situation in the text. Students have to find solutions to the job. They may have other sources to 

make their idea more potent and more effective. In this case, the researcher uses action verbs to 



Riski Norita Sari, Zulfa Sakhiyya/ ELT Forum 9 (2) (2020) 

210 

 

 

decide because this question encouraged students to determine what the proper actions are. Both of the items 

use the cognitive process of critiquing. According to Anderson and Krathwohl (2001), critiquing is judging 

because those questions demand students to give an opinion.   

 

Creating level 

The creating level is the highest at Bloom's Taxonomy level. From this level, students have to result in a new 

product. The new product can be managed from the pattern or information before (Anderson & Krathwohl, 

2001). This level can be done after students through other levels such as remembering, understanding, 

applying, analyzing, and evaluating. Creating a level is the highest level in Bloom's Taxonomy level. From 

this level, students have to result in a new product. This level is done after students through other standards 

such as remembering, understanding, applying, analyzing, and evaluating.  

Yet creating a level is the part of a Higher-Order level, so students have to think critically to manage 

this case. Moreover, in creating something, students have to collect from some sources to make the product 

in prior knowledge. Some elements can be put in a way that makes the product to be structured well. 

Subsequently, students should have ideas and creativity. Also, this level adopted three cognitive processes 

such as generating, planning, and producing.  Based on the research from this book, the researcher found 

only one question that requires creating a level. As a result, this is only 1% of the total. Creating a degree is 

the lowest number found in the reading task of Symphony 1. The following is the creating level found 

in Symphony 1.   

 

Table 8.The types of creating items in Symphony 1 
No Item Types Frequency Percentage (%) 

1.  Rearranging information related to the text  1 100 

2.  Summarizing texts  0 0 

3.  Writing or retelling the material of the text using his own words  0 0 

4.  Predicting events or solutions related to the text  0 0 

5.  Discussing to persuade  0 0 

6.  Hypothesizing data  0 0 

7.  Generating information related to the text  0 0 

8.  Combining his information with the information in the text  0 0 

9.  Connecting knowledge from different sources  0 0 

Total 1 100 

 

The data from Table 8 shows that the Lower-Order Level dominates the task. It can be seen from 

Figure 1 that Lower-Order Level was 63%. This result is significant because this is more than 50%. The 

research also found that there is no level of applying found in the data in the Lower-Order level. Higher-Order 

Level in the tasks is dominated by analyzing level it reaches 32% out of 37% from the total of Higher-Order 

Level. It is followed by evaluating level counted 4%, then creating a level of only 1%.  

 

Table 9.The example of creating item in Symphony 1 
No Item Categories Action 

Verbs 

Cognitive 

Process 

1. Compose a single stanza quintet using the 

same pattern of rhymes found in this poem. 

Creating adapt  Generating 

 

This question demands students to make a new product by using the pattern in the issue. As a result, 

this case encourages students to adapt further information. Also, the cognitive process involved in this 

question is generating. Anderson and Krathwohl (2001) declared that the generating process involved with 

produce or create something.  

 The Higher-Order Level exists on the task, but most of them were not distributed well in all items by 

Kesthta and Sheif (2013). It can be seen from the data of analyzing level which only contains five out of 

thirteen pieces. It consists of comparing items in the text, guessing the meaning of words in context, reading 

between the lines, explaining the information introduced, and recognizing clues and evidence. Besides, most 

questions require revealing the information presented that reached 42% out of 100%. On the other hand, 

guessing the meaning of words, only 2% out of 100%. This result is quite a significant comparison. It 



Riski Norita Sari, Zulfa Sakhiyya/ ELT Forum 9 (2) (2020) 

211 

 

 

demonstrates that the tasks in the textbook did not have many varieties of Higher-Order Level, and this is in 

line with the statement that most of the textbooks in Indonesia contain Lower-Order Thinking Skills because 

most of the books comprise of memorizing and understanding (Margana & Widyantoro, 2017). Thus, based 

on the finding, this book could not stick to the criteria of Higher-Order. The ideal distribution of Higher-

Order and Lower-Order level is 50:50 (Anggraeni & Suharyadi, 2013). If this continues, then students will 

not be exposed to Higher-Order instructions that stimulate their critical thinking and holistic comprehension.  

 

CONCLUSION 

The researcher can conclude some numbers from the overall findings. First, the higher-order cognitive skills 

in the textbook's reading tasks are not distributed and treated well. The result of Higher-Order levels in the 

textbooks' reading tasks was less than Lower-Order levels. According to the information, there are six 

classifications of the question items from the cognitive domain of Revised Bloom's Taxonomy. Also, Lower-

Order levels dominated the reading tasks in Symphony 1 with the percentage of 63%. Remember, the 

standard has the highest frequency among all levels, with 50 cases and a 37% rate. The understanding level 

resulted in 32 cases or 24%. 

Furthermore, the Lower-Order level apply does not exist on the reading tasks. In another case in the 

information, there are 39% of higher-order level questions, or there are 50 questions from the total. Then, 

the Higher-Order level of analyzing reaches 32% or 43 items—meanwhile, the higher-order level of 

evaluating touched 4% or six questions. Subsequently, the Higher-Order level creating level recorded only 

1% or one issue. As a result, creating is the lowest point from the higher-order level, and analyzing is the 

highest point of the higher-order level. 

Then questions indicated of Higher-Order Level have been analyzed to know the variety of HOTS 

found in the textbooks. From analyzing the level contained four out of thirteen items. They compared things 

in the text, guessing the meaning of words in context, reading between the lines, explaining the information 

introduced, and recognizing clues and evidence. Also, most questions that contain disclosing the 

information presented reached 42% out of 100%. On the other hand, the questions that contain guessing the 

meaning of words had only 2% out of 100%. The result was entirely a significant comparison. It proved that 

the tasks in the textbook do not have many varieties of Higher-Order Level. 

 

REFERENCES 

Abdelrahman, M. S. H. B. (2014). An analysis of the tenth grade English language textbooks questions in 

Jordan based on the revised edition of Bloom's taxonomy. Journal of Education and Practice, 5(18), 139-

151.  

Adams, N. E. (2015). Bloom's taxonomy of cognitive learning objectives. Journal of J Med Lib Assoc, 103(3), 

152-153. 

Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of 

Bloom's taxonomy of educational objectives. Longman 

Anggraeni, A., & Suharyadi, S. (2013). Reading questions in English textbooks for senior high schools grade 

X. English Language Education, 3(1). 5-16 

Antara. (2019, December 3). Skor PISA murid Indonesia di bawah rata-rata, Ini kata Nadiem. Tempo.co. 

https://tekno.tempo.co/read/1279711/skor-pisa-murid-indonesia-di-bawah-rata-rata-ini-kata-

nadiem 

Assaly, I., & Igbaria, A. K. (2014). A content analysis of the reading and listening activities in the EFL 

textbook of Master Class. A Content Analysis of the Reading and Listening Activities in the EFL Textbook of 

Master Class, 3(2), 24-38. https://doi.org/10.11648/j.edu.20140302.11 

Assaly, I. R., & Smadi, O. M. (2015). Using Bloom's taxonomy to evaluate the cognitive levels of Master 

Class textbook's questions. Journal of English Language Teaching, 8(5), 100-110. 

Crescentini, A., & Mainardi, G. (2009). Qualitative research articles: Guidelines, suggestions, & needs. 

Journal of Workplace Learning, 21(5), 431-439. 

Freahat, N. M., & Smadi, O. M. (2014). Lower-order & higher-order reading questions in secondary & 

university level EFL textbooks in Jordan. Theory & Practice in Language Studies, 4(9), 1804-1813. 

Holsti, O. R. 1969. Content analysis for the social sciences and humanities, reading. MA Addison Wesley 

Keshta, A. S., & Seif, A. (2013). Evaluating the higher order thinking skills in reading of English for Palestine 

grade eight. Asian Journal of Education and E-Learning, 1(1), 2321–2454. 

http://library.iugaza.edu.ps/thesis/105688.pdf%0Awww.ajouronline.com 

Manion, L., Cohen, L., & Morrison, K. (2007). Research methods in education (6th ed). Routledge Falmer. 

https://tekno.tempo.co/read/1279711/skor-pisa-murid-indonesia-di-bawah-rata-rata-ini-kata-nadiem
https://tekno.tempo.co/read/1279711/skor-pisa-murid-indonesia-di-bawah-rata-rata-ini-kata-nadiem
https://doi.org/10.11648/j.edu.20140302.11
http://library.iugaza.edu.ps/thesis/105688.pdf%0Awww.ajouronline.com


Riski Norita Sari, Zulfa Sakhiyya/ ELT Forum 9 (2) (2020) 

212 

 

 

Margana, M., & Widyantoro, A. (2017). Developing English textbooks oriented to higher-order thinking 

skills for students of vocational high schools in Yogyakarta. Journal of Language Teaching and Research, 

8(1), 26-38 

Milles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Sage 

Publications Inc. 

Nourdad, N., Masoudi, S., & Rahimali, P. (2018). The effect of higher-order thinking skills instruction on 

EFL reading ability. International Journal of Applied Linguistics & English Literature, 7(3), 231- 237. 

Ospina, S. (2004). Qualitative research. Thous& Oaks CA. 

PISA for Development Reading Framework. (2018). 25–48. https://doi.org/10.1787/9789264305274-4-en 

Raqqad, Y. A., & Ismail, H. H. (2018). Analyzing the reading questions of AP12 textbook according to 

Bloom’s taxonomy. International Journal of Education, Psychology & Counseling, 3(22), 84-94. 

Sherman, R. R., & Webb, R. B. (2005). Qualitative research in education: Focus and methods. Falmer Press  

Tangsakul, P., Kijpoonphol, W., Linh, N. D., & Kimura, N.  (2017). Using Bloom’s Revised Taxonomy to 

analyze reading comprehension questions in team up in English 1-3 & grade 9 English O-net tests. 

International Journal of Research Granthaalayah, 5(7), 31-41. 

Thamrin, N. R., & Agustin, S. (2019). Conceptual variations on reading comprehension through higher-

order thinking skills (HOTS) strategy. Journal of English Education, 7(2), 93-100. 


