Language Circle: Journal of Language and Literature 15 (1) October 2020 83-89 Available online at http://journal.unnes.ac.id/nju/index.php/LC P-ISSN 1858-0157 E-ISSN 2460-853X An Analysis of Higher Order Thinking Skills Realization in Reading Comprehension Questions Ervina Juli Aryani, Sri Wahyuni* Universitas Negeri Semarang, Indonesia *Email: sriwahyunifbs@mail.unnes.ac.id DOI https://doi.org/10.15294/lc.v15i1.26064 Submitted 5 September 2020. Revised 30 September 2020. Accepted 23 October 2020 Abstract The significance of Higher Order Thinking Skills (HOTS) has led many educational stakeholders to include the skill in educational curriculum and more specifically in the subjects that students learn. In the context of learning English, HOTS should be instilled in four language skills, especially reading skill. This skill is vital since students are demanded to be critical of what they read. Therefore, this research aimed at investigating how the reading comprehension questions in English textbooks for three different grades (X, XI, and XII) published by the Minis- try of Education and Culture facilitate HOTS. By employing cognitive level of Bloom’s revised taxonomy (2001) and critical thinking task type of Numrich’s sequence (Beaumont, 2010), the questions are analyzed qualitatively. The results reveal that HOTS are realized adequately in the textbook for grade XI. In the textbooks for grade X and XII, HOTS are found in small quantity. Besides, based on Numrich’s sequence, the majority of the ques- tions focus on the text which means that critical thinking-based questions are found in small number. Thus, it is expected that this result assists teachers, textbook writers, and future researchers to engage more on the imple- mentation of HOTS. Keywords: critical thinking; English textbooks; HOTS; reading comprehension questions thinking. It is explained that HOTS are defined into three categories namely transfer, critical thin- king, and problem solving (Brookhart, 2010, p.3). Based on this categorization, it is defined that HOTS in terms of transfer demand students to apply knowledge and skills they learn in a new context that they do not repeatedly experience. In terms of critical thinking, HOTS encourage the skill of making judgement and making decision. While on the problem solving, students are ex- pected to formulate solutions to certain problems. Therefore, in accordance with the definitions, HOTS can be assumed as skills of engaging with unfamiliar context, the activity of making judge- ment and decision, and also solving problems. In connection with the categorization, it is inferred that through the emphasis of HOTS, there is an effort of promoting critical thinking. In connecti- on with the insertion of HOTS in the educational field, it directly affects the design of teaching- learning activities and textbooks in facilitating HOTS. It is expected that those components pro- vide wide opportunities for students to exercise HOTS. Moreover, textbook takes an integral part in promoting HOTS since it becomes a universal element of ELT teaching (Hutchinson & Torres, 1994). It is widely used considering the benefits INTRODUCTION The great challenges and rapid develop- ment of science and technology demand people to possess the skills needed in this era in order to win those challenges and demands. The skills are formulated under the name of 21st century skills which include critical thinking and problem solving, creativity and innovation, collaboration, question formulation, global awareness, commu- nication skills, and technology skills (Harvard Advanced Leadership Initiative, 2014). Those skills are perceived to be the provisions to face the global challenges in these days. Noting the significance of the skills, there is an initiation of the inclusion of the 21st century skills in many educational curricula. This attempt is also projected in the Indonesia national educa- tion curriculum which emphasizes Higher Order Thinking Skills (HOTS) as one of the competen- cies that are envisioned (Pusat Penelitian Kebijakan Pendidikan dan Kebudayaan, Badan Penelitian dan Pengembangan, Kementerian Pendidikan dan Kebu- dayaan, 2018). The inclusion of HOTS is believed as the effort of fostering critical thinking since it is in line with the statement by Brookhart (2010) regarding the relation between HOTS and critical Language Circle: Journal of Language and Literature 15 (1) October 2020 83-89 84 can practice, see their progress, and even develop HOTS. However, based on my previous studies, it is found that HOTS-based questions are reali- zed in small quantity. Another result obtained by Ulum (2016), Barashid (2020), Tangsakul (2017), and Permatasari (2012) revealed that most of the reading comprehension questions are tapped by the lower categories and less likely promote higher categories. Ulum (2016) who conducted a descriptive content analysis of a reading com- prehension questions in a course book Q: skills for success 4 reading and writing according to Bloom’s taxonomy revealed the same findings that the reading comprehension questions were lack of HOTS. These results urge the action to assure the implementation of HOTS in reading comprehension questions. The results of some previous researches bring us evidences that textbooks play an impor- tant part in supporting the development of HOTS, and reading comprehension questions should ide- ally implement HOTS. However, based on pre- vious studies HOTS are realized in small num- ber. Connecting these facts, it is essential to make sure that the elements of teaching-learning activi- ty, such as, textbook and reading comprehension questions should implement HOTS. Therefore, this present research is intendedly conducted to analyze how HOTS are realized in the reading comprehension questions found in English tex- tbooks using cognitive level of Bloom’s revised taxonomy (2001), and task types of Numrich’s sequence (Beaumont, 2010). The research is mainly formulated to answer five main questions: (1) how are HOTS realized in reading compre- hension questions found in the textbook for grade ten?; (2) How are HOTS realized in the reading comprehension questions found in the textbook for the eleventh grade?; (3) How are HOTS rea- lized in the reading comprehension questions found in the textbook for the twelfth grade?; (4) How are the reading comprehension questions found in the textbooks for the tenth, eleventh, and twelfth grade realized critical thinking task of Numrich’s sequence?; (5) How are the differences among those three textbooks in realizing HOTS and critical thinking of Numrich’s sequence in reading comprehension questions? METHODS This research is a content analysis ana- lyzing the content of textbooks, specifically the reading comprehension questions found in the analyzed textbooks. Since this present research aimed at describing the analysis result regarding that exist not only for students, but also for teach- ers. For students, textbook provides several com- ponents such as materials, activities for students’ practice, and syllabus (when reflected the objecti- ves that have been determined) (Cunningsworth, 1995, p.7). In addition, textbook also provides components that ease teachers in deciding what the most appropriate things for their classes are (Bojanic & Topalov, 2016). This is strengthened by Herlinda (2014) stating that textbooks play a crucial role in teaching-learning activity. It is revealed that both teachers agreed that textbook provided materials and activities aspects in lan- guage learning, guided teachers in giving home- work and created standardized instruction, and also gave idea in making lesson plan. It was also found that in teaching-learning activity, both te- achers use more than one textbook as reference. This shows that textbooks play a vital role in clas- sroom activity. Realizing those significant roles; therefore, textbooks should ideally come with materials and exercises fostering HOTS in any subjects, inclu- ding English. HOTS should be realized in all of the language skills, specifically reading skill sin- ce it is extremely required nowadays. Moreover, in current days, students are not only demanded to be able to read, but are also demanded to be able to make meaning of the information they gain (Wahyuni, 2019) and further to be critical of what they read (Taylor, 2018). Referring to the importance of making meaning from reading text; thus, it is no wonder that great effort of assessing and exploring stu- dents’ performance on the reading skill has been made. For instance, international test, namely PISA, has been conducted in order to assess pro- ficiency on reading, mathematics, science and an innovative domain, and students’ well-being (OECD, 2018). Referring to the result of the test held on 2018, it is revealed that Indonesian students scored lower than the average score of OECD in reading. Translating the score, 30% of Indonesian students attain at least level 2 of proficiency which primarily deal with identifying main idea in a moderate length, find information based on explicit, though sometimes complex cri- teria, and reflect on the purpose and form of texts when explicitly directed to do so (OECD, 2018). Some a negligible percentage reach top perfor- mers. These impliedly means that further attenti- on on the reading proficiency should be put. Taking the issues into consideration, there should be created exercises and specifically samp- le of reading comprehension questions promo- ting HOTS. By providing these aspects, students Language Circle: Journal of Language and Literature 15 (1) October 2020 83-89 85 the realization of HOTS in reading comprehensi- on questions using cognitive level of Bloom’s re- vised taxonomy (2001) and critical thinking task types of Numrich’ sequence (Beaumont, 2010), it is categorized into descriptive qualitative rese- arch. The reading comprehension questions un- der the study are those of questions following reading passages found in the textbooks entitled Bahasa Inggris SMA/MA/SMK/MAK kelas X, XI, XII Edisi Revisi 2017 published by the Ministry of Education and Culture. There are 170 reading comprehension questions in the textbook for gra- de X, 42 reading comprehension questions in the textbook for grade XI, and 117 reading compre- hension questions in the textbook for grade XII which were examined. Since the data under ana- lysis are in the sort of written text, thus, it is cate- gorized as qualitative data. In collecting the data, documentation was chosen with three main steps: (1) reading the tex- tbooks under analysis; (2) collecting the reading comprehension questions; and (3) documenting the questions. Having been collected, then, the data were analyzed. In analyzing the data, a frame- work of data analysis by Miles and Huberman (1994, p.10) was employed. Accordingly, there are three major steps of data analysis: (1) data reduction; (2) data display; (3) conclusion and verification. Data reduction is the stage of orga- nizing the raw data into the data which are ready for further analysis and conclusion drawing. Data display is believed as the stage of displaying the compressed data. Lastly, conclusion and verifi- cation is the stream of concluding the displayed data and then verifying the conclusion. Referring to the framework, the data on this present rese- arch were analyzed with the following procedu- res: (1) sorting relevant reading comprehension questions and listing the questions based on the textbook they belonged to; (2) identifying the data and displaying the result of identification in the form of table and narrations; (3) drawing con- clusion and validating the conclusion. In order to validate the conclusion, trian- gulation was conducted. It is employed in order to show that other independent measures support the findings or at least don’t contradict (Miles & Huberman, 1994:266). In doing triangulation, triangulation by data source and theory were employed. Therefore, two English teachers were invited to share their perspectives on the imple- mentation of HOTS in reading comprehension questions in English textbooks trough question- naire, and related previous researches were se- lected to triangulate the analysis result. The pro- cedure of analysing data is visualized in Figure 1. Figure 1. Data analysis process RESULTS AND DISCUSSION As previously mentioned that this research specifically investigates the realization of HOTS in reading comprehension questions in three Eng- lish textbooks for different level of grades using cognitive level of Bloom’s revised taxonomy and critical thinking task types of Numrich’s sequen- ce (Beaumont, 2010), it is revealed that HOTS are realized in three different ways. In general, HOTS are realized in the three textbooks, however, the realization is in different number. For more detail result, Table 1 visualizes the frequencies and per- centages from the three textbooks. Citing the frequencies and the percenta- ges of the realization of cognitive level shown in Table 1, there are three main points that can be derived: first, in the textbook for grade X, the Table 1. Analysis result based on cognitive level of Bloom’s revised taxonomy (2001) Cognitive level Frequency on each Grade of the book X % XI % XII % Remember 125 73.52% 8 19% 72 61.53% Understand 22 12.94% 8 19% 25 21.36% Apply 0 0 0 0 1 0.85% Higher order thinking skills Analyze 9 5.29% 11 26.19% 10 8.54% Evaluate 11 6.47% 12 28.57% 6 5.12% Create 3 1.76% 3 7.14% 3 2.54% Language Circle: Journal of Language and Literature 15 (1) October 2020 83-89 86 categories on the cognitive level are not equally distributed. The lower categories tap the ques- tions in comparison to the higher categories. Although applying category is not found, re- membering and understanding are found in high percentages. Remembering becomes the major category found with more than three quarter of the total questions. In dealing with this category, students mostly cope up with explicit information written in the text, e.g. Does she like animals? What impression do you get when you read the word majes- tically? In accomplishing the questions, students only need to read the text and scan the related information. The answers are explicitly written in the text. This type of question is found in more than three quarter of the total questions, which means that students mostly deal with the questi- on focusing on the lower category. This phenomenon makes the realization of the higher categories in lower percentages. Lin- king to the frequencies of HOTS-based questions, there are only 13.52% of HOTS based questions. These HOTS-based questions are distributed into analyzing 9 questions, evaluating 11 questions, and creating 3 questions. Through those type of questions, students are encouraged to deal with the skill to analyze information on the text by differentiating the elements found there, such as, Read the text 1 again. Find out the similarities between text 1 and text 2. This question demands students to break the text and analyze the elements to find out the similarities and differences. Another skill to give evaluative judgement is also manifested in the question like, Do you think that the information in the text is clear? This question requires students to evaluate the information in the text and then give their judgement on that information based on the imposed criteria. An important skill to cre- ate is also emphasized through 3-unit questions. In the skill of creating, students are demanded to be able to hypothesize, make a plan, or create a product, for instance, Why do you think the date of the battle of Surabaya is used as a momentum to commemorate our hero’s contributions? Through this question, students need to deal with the process of hypothesizing premises. Second, the reading comprehension ques- tions found in the textbook for grade XI are do- minated by HOTS. This is proven by the availabi- lity of HOTS in more than 50% of the questions. Based on the analysis, there are 61.9% of the reading comprehension questions which are ca- tegorized into HOTS. There are 26.19% of ana- lyzing category, 28.57% of evaluating category, and 7.14% of creating category. From the three categories, evaluating is found higher than analy- zing and creating. In evaluating, students mostly deal with making judgement and checking in- consistency. This skill becomes one of the main concerns in critical thinking in which students see an issue from various perspectives and positions (Parviz & Imam, 2018). This skill is important for the development of critical thinking for stu- dents. Hence, a great number of questions in this category could stimulate and help students train HOTS ability. The great number of HOTS-based questi- ons proves that students are given wider opportu- nities to practice higher thinking skills. This can be seen through the manifestation of analyze, evaluate, and create in a more than half of the to- tal questions. For instance, these skills are found in the questions, for examples, Do you think the poet is content with his choice? Give reasons to support your answer!; Do you think smoking is dangerous?; Do you think it should be banned?; If you had to chan- ge the lyrics of stand by me, which lyrics would you change?; Who is your hero?; What might the two roads represent or symbolize? Make a list of possibilities and discuss with your partner! These questions promote the process of investigating writer’s point of view, making judgement, and hypothesizing. These processes become one of the concerns mainly emphasized in higher order thinking skills. Third, reading comprehension questions found in the textbook for grade XII are domi- nated by lower category, which is remembering category. This creates a situation in which the higher categories are only attained in small quan- tity. From overall 117 questions, HOTS-based questions only appear in 19 out of 117 questions or in 16.2% of the questions. These questions are distributed into analyzing 8.54%, evaluating 5.12%, and creating 2.54% of the questions. In comparison to LOTS, HOTS are found in lower percentages. In HOTS-based questions, the skill is trans- lated into the question such as What is being de- bated? What is the fact? Why do you think so?; What differentiate one type of bullying from another?; do you think is the most damaging kind of bullying?; If you were one of the parents, what would you do to respond to some problems with the online system? In the first and second questions, students are required to de- monstrate the skill of analyzing by distinguishing facts from opinions, and also differentiating one element to another. While the third question, stu- dents are given a challenge to give judgement over kind of bullying. What students are expected to do is deciding which kind of bullying that fits the label of the most damaging, considering all the features mentioned in each type of bullying. On Language Circle: Journal of Language and Literature 15 (1) October 2020 83-89 87 the other hand, the last question asks students to formulate solutions to solve online system prob- lem. In this task, students must produce a soluti- on to overcome the problems. This task belongs to creating category in which students mainly fo- cus on connecting elements to create a functional whole (Anderson & Krathwohl, 2001, p. 84). The cognitive level of Bloom’s revised ta- xonomy (2001) is created for the purpose of assis- ting teachers in designing tasks based on the cog- nitive level that is aimed to be reached (Anderson & Krathwohl, p.6). The categories on the cogniti- ve level work hand in hand. The lower categories need to be emphasized to support the mastery of the higher categories. The higher categories are those the skills of analyzing, evaluating, and cre- ating. In relation to that, only reading compre- hension questions in the textbook for grade XI provide adequate number of questions that are qualified to HOTS. This is claimed as adequate since the realization of HOTS is found in more than 60% of the questions. The frequent number of HOTS gives students more platform to shar- pen their ability on solving problems requiring higher degree of thinking. However, the reading comprehension questions found in the textbooks for grade X and XII, majorly focus on the lower categories, es- pecially remembering. This category is good to recall the previously learnt information, but too many questions of this category is not enough to promote HOTS. In these two textbooks, HOTS- based questions are only found in less than 20% of the questions which is not sufficient to stimula- te and further progress the ability of HOTS. This brings the evidences that HOTS ability should be developed. This development can be realized by familiarizing the skills for students. One thing that can provide the platform to familiarize and train the ability of HOTS is textbook (Pratama & Retnawati, 2018). Thus, in fostering HOTS for pursuing critical thinking as how it is mentioned in the 21st century skills, HOTS-based questions should be ideally realized more often (Wu & Pei, 2018). To sum up, two major findings emerge from this present research. The reading comprehension questions found in the textbooks for grade X and XII mostly deal with the lower categories than the higher categories. HOTS-based questions are pre- sented in a small number of questions. These are in line with the result obtained by Ulum (2016), Brasahid (2019), and Tangsakul (2017). In those researches, reading comprehension questions are still lack of HOTS-based questions. In compari- son, there is an initiation of imposing HOTS in reading comprehension questions in adequate number. This is realized in the reading compre- hension questions found in the textbook for grade XI. This is supportive with the research by Setiy- awati who unveiled that HOTS are found domi- nant in the textbook she investigated. These two findings are confirmed by two English teachers, who are the users of the textbooks. They stated that HOTS are moderately realized in the three textbooks. This means that HOTS, are realized, but not in sufficient number. Aside of HOTS analysis, the analysis based on task types of Numrich’s sequence is also conducted. The result of the analysis is displayed in Table 2. Table 2 shows the result of analysis based on task types of Numrich’s sequence. There are seven tasks which are categorized into three per- spectives. Those tasks are the sequence in build- ing critical thinking starting from investigating personal views, engaging with the main text, and focusing beyond the text (Beaumont, 2010). From Table 2, it is presented that the task types are not equally realized. From the three tex- tbooks, observing and inquiring further are not found. In contrast, understanding and organi- zing, and interpreting are dominated. Understan- ding and organizing task is found in more than 50% of the questions in the textbooks for grade X and XII. Besides, interpreting task is realized Table 2. Analysis of reading comprehension questions based on Numrich’s sequence Perspective Cognitive level Frequency on each Grade of the book X % XI % XII % Focus on stu- dents’ world Observing 0 0 0 0 0 0 Identifying assumptions 7 4.11% 8 19% 4 3.41% Focus on the text Understanding and organizing 124 72.94% 5 11.9% 80 68.37% Interpreting 27 15.88% 18 42.85% 22 18.8% Focus beyond the text Inquiring further 0 0 0 0 0 0 Analyze and evaluate 10 5.88% 9 21.42% 8 6.83% Making decision 2 1.17% 2 4.76% 3 2.56% Language Circle: Journal of Language and Literature 15 (1) October 2020 83-89 88 in nearly 50% of the questions in the textbook for grade XI. The domination of understanding and organizing, and interpreting shows that most of the questions lead students to focus on the infor- mation on the text. In understanding and organizing task, the answer can be directly picked from the explicit ex- cerpt on the text. This task is part of critical thin- king, however, in dealing with the task, students do not need to think critically since the answer is already written on the text. This is slightly dif- ferent from interpreting task. This task demands students to focus on the information in the text, but students are required to understand the infor- mation deeper. They do not rely on the literal sta- tements, but they have to investigate the meaning behind the literal information. However, in this task, students still focus on the text and not yet move to think beyond the text. Seeing the percentages, it can be stated that the tasks focusing beyond the text are distributed in lower percentages. In the textbooks for grade X and XII, questions focusing beyond the text are only found in less than 10% of the questions. Mo- reover, inquiring further is not realized, and ma- king decision is only found in 2 question items. This also happens in the textbook for grade XI. However, in this textbook, the percentages of the activity focusing beyond the text is higher with 26.18% of the questions. Those representations share similarity in terms of focusing on the text. The difference is on the task the questions mostly focus on. In the tex- tbooks for grade X and XII, understanding and organizing task is dominating. In the textbook for grade XI, the majority of the questions belong to interpreting. Deducing the information related to the task types of Numrich’s sequence in the three textbooks, it is discovered that the reading com- prehension questions primarily focus on the text. This perspective is surely part of critical thinking; however, dealing with this perspective too much is not enough to stimulate critical thinking. Mo- reover, focusing on evaluating, analyzing, making decision, and problem solving, which are impor- tant in critical thinking, are only found in small quantity. According to Paul and Elder (2008) citing in Parviz and Imam (2013) critical thinking is a matter of analyzing and evaluating in the purpo- se of improving it. Critical thinking is a kind of thinking involved in solving problems, designing inferences, calculating likelihoods, and making decision (Parviz & Imam, 2013). In comparison to the definitions, from three analyzed textbooks, the task of analyzing, evaluating, finding more sources of information, and making decision are found in small percentages. Inquiring further, which focuses on finding out more information is not found in the three textbooks. Analyzing and evaluating is found in less than 6% of the questi- ons in the textbook for grade X, 21% of the ques- tions in the textbook for grade XI, and less than 7% of the questions in the textbook for grade XII. Furthermore, making decision task is only found in 2 questions in the textbooks for grade X and XI, and 3 questions in the textbook for grade XII. Inferring the percentages, the critical thinking- based questions are found in minimum percenta- ges. In comparison to that, critical thinking seems to be imposed in only small quantities and needs more number in order to pursue the skill. In connection to critical thinking task ty- pes of Numrich’s sequence, it is explained that critical thinking is built by experiencing various activities that give increasing challenges. It starts from investigating students’ own world, focusing on the text, and then focusing beyond the text (Beaumont, 2010). It is also explained that criti- cal thinking activates emotional or get responses to finally create a reasonable judgement (Beau- mont, 2010). However, it is revealed that the rea- ding comprehension questions only focus on the text. The activity of investigating students’ perso- nal emotion and then connect it to think beyond the text is somewhat neglected. Thus, it can be said that the three textbooks are less likely boost critical thinking since the tasks that students per- form do not call students to think critically to en- gage with the world beyond the literal informa- tion that they gain in the text and the problem beyond what they solve in the text. Those results are agreed by the findings ob- tained by Permatasari (2010) which released the information regarding the tasks students mainly focus on. In that research, the majority of the questions push students to focus on the text. This means that the attention on developing critical thinking-based questions should be increased. CONCLUSIONS In accordance with the presented findings, it is drawn that HOTS are only adequately reali- zed in the textbook for grade XI. In the textbooks for grade X and XII, HOTS are found there, but in small quantity. This is quite surprising since HOTS-based questions are demanded in current days by the curriculum. Furthermore, it is also explained that critical thinking-based questions according to task types of Numrich’s sequence Language Circle: Journal of Language and Literature 15 (1) October 2020 83-89 89 appear in low percentages, for the questions com- monly focus on the text. Therefore, taking the findings into con- sideration, it is advisable that education stake- holders, teachers, textbook writers, and house of publishers should give more attention on the effort of fostering and developing HOTS. It is ex- pected that more sources on HOTS-based questi- ons should be added. Further action in engaging with the issue on the development of HOTS are expected to be initiated. REFERENCES Anderson, & Krathwohl. (2001). A taxonomy for learn- ing, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Longman Barashid, M. (2020). Analyzing the Content of Read- ing Texts Questions in Flying High for Saudi Arabia. International Education Studies, 13(6), 96-103. Beaumont, J. (2010). A sequence of critical thinking tasks. Tesol Journal, 1(4), 427-448. Radić-Bojanić, B. B., & Topalov, J. P. (2016). Textbooks in the EFL classroom: defining, assessing and analyzing. Zbornik radova Filozofskog fakulteta u Prištini, (46-3), 137-153. Brookhart, S. M. (2010). How to assess higher-order think- ing skills in your classroom. ASCD Cunningsworth, A. (1995). Choosing your coursebook- Macmillan Heinemann Publishers Ltd Brown, S. (2014). education for the 21st century: ex- ecutive summary.© Advanced Leadership Ini- tiative. Business Administration Harvard Business School. Cambridge. UK, 14. Herlinda, R. (2014). The use of textbook in teaching and learning process (a case study of two EYL teachers). The 61st TEFLIN International Confer- ence, UNS Solo. Hutchinson, T., & Torres, E. (1994). The textbook as agent of change. ELT journal, 48(4) Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). Sage publications OECD. (2019). Programme for international students assessment (PISA) results from PISA 2018, 1(3) Parviz, B., & Imam, A. (2012). Manifestation of critical thinking skill in the English textbooks employed by language institutes in Iran. Inter- national Journal of Research Studies in Language Learning, 2(2), 27-38 Permatasari, R. (2012). The Critical Thinking Tasks Dis- played on the Textbooks for Eleventh Grade Students of Senior High School (Doctoral dissertation, Universitas Negeri Semarang). Pratama, G. S., & Retnawati, H. (2018, September). Urgency of higher order thinking skills (HOTS) content analysis in mathematics textbook. In Journal of Physics: Conference Series (Vol. 1097, No. 1, p. 012147). Pusat Penelitian Kebijakan Pendidikan dan Kebuday- aan, Badan Penelitian dan Pengembangan, Kementerian Pendidikan dan Kebudayaan. (2018). Muatan HOTS pada Pembelajaran Kurikulum 2013 Pendidikan Dsar. Rahma, D. N. (2019). An analysis of reading compre- hension questions made by English teacher at SMAN 2 Sidoarjo based on Baret’s taxonomy iNELTAL Conference Proceedings. The Internation- al English Language Teachers and Lecturers Confer- ence. Setiyawati, E. (2016) using revised Bloom’s taxonomy to analyze reading comprehension question in English textbook entitle ”Bahasa Inggris” for grade XI by Kemendikbud 2014. Unpublished thesis. Universitas Katolik Widya Mandala, Surabaya. Tangsakul, P., Kijponphol, W., Linh, N. D., Kimura, L. N. (2017). Using bloom’s revised taxonomy to analyze reading comprehension questions in team up in English 1-3 and grade 9 English o-net test. International journal of research- granthaalayah, 5(7), 31-41 Taylor, S. (2018). Reading for meaning: a guide to the re- search on best-practice teaching of comprehension in today’s world. Oxford university press Ulum, O. G. (2016). A descriptive content analysis of the extent of bloom’s taxonomy in the reading comprehension questions of the course book Q: skill for success 4 reading and writing. The qualitative report, 21(9), 1674-1683. Wahyuni, S. (2019). Investigating students’ critical lit- eracy: a preliminary study. 2nd UNNES TEFLIN seminar Wu, Y., & Pei, Z. (2018). An investigation of critical thinking manifested in the questions of EFL textbooks for tertiary-level English majors of China. American Journal of Education and Learn- ing, 3(2), 72-84.