The particular dialect or language that a person chooses to use on any occasion is called a code 1014 Copyright © 2023 The Author IDEAS is licensed under CC-BY-SA 4.0 License Issued by English study program of IAIN Palopo IDEAS Journal of Language Teaching and Learning, Linguistics and Literature ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) Volume 11, Number 2, December 2023 pp. 1014 - 1030 Assessing Students’ Higher-Order Thinking Skills In Reading Comprehension Syah Ibi Hamma1, Syamsiarna Nappu2,Erwin Akib3 syehibi@icloud.com1, syamsiarna.nappu@unismuh.ac.id2, erwin.akib@unismuh.ac.id3 1Magister Pendidikan Bahasa Inggris, Universitas Muhammadiyah Makassar 2 Magister Pendidikan Bahasa Inggris, Universitas Muhammadiyah Makassar 3 Magister Pendidikan Bahasa Inggris, Universitas Muhammadiyah Makassar Received: 2023-01-10 Accepted: 2023-07-03 DOI: 10.24256/ideas.v11i2.4182 Abstract This research aimed to find out the score of students’ higher-order thinking skills (HOTS) in Reading Comprehension and the challenges faced by the students when solving HOTS. This research used a quantitative descriptive design. The data were obtained by using tests and questionnaires. The subjects of this research were 22 students who studied in the Twelfth Grade IPA 1 in SMA Pesantren IMMIM Makassar. By definition, Higher-Order Thinking is divided into 3 Categories which were Higher-Order Thinking As Transfer (C4), Higher-Order Thinking as Critical Thinking (C5), and Higher-Order Thinking as Problem Solving (C6). The students answered the test and the questions were at the HOTS level. In challenges, this research investigated the challenges faced by the students when solving HOTS by analyzing the questionnaire. The findings of this research revealed that the score of students’ HOTS was 64,54% which was considered Good, It shows that students scored Higher-Order Thinking as Transfer (C4) was 81,8%, while Higher-Order Thinking as Critical Thinking (C5) was 54,5% and Higher-Order Thinking as Problem Solving (C6) was 9.1%, and as for the challenges faced by students when solving HOTS was 16.63% students facing difficulties in understanding the Meaning of Word (C4), 27.72% facing difficulties in Making Inferences (C5) and 28.81% students facing difficulties in Determining the Main Idea (C6) when solving HOTS Questions. It can be concluded that the student’s Higher-Order Thinking as Transfer (C4) was Excellent, Higher-Order Thinking as Critical Thinking (C5) was good and http://u.lipi.go.id/1457703302 mailto:syehibi@icloud.com mailto:syamsiarna.nappu@unismuh.ac.id IDEAS, Vol. 11, No. 2, December 2023 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 1015 Higher-Order Thinking as Problem-Solving was Poor, and the challenges that the students faced when solving HOTS Questions were mostly determining the main idea and making inference. Keywords: Higher-Order Thinking Skills, Reading Comprehension, and Reading Assessment Introduction The only foreign language taught in the Indonesian K13 Revision curriculum is English. Reading, hearing, writing, and speaking are all important components of learning English as a foreign language, in accordance with Indonesian Education Ministry regulation No. 22 of 2006 governing elementary and middle school standards. Learning English is more difficult than most people realize, and Indonesians do not speak English natively. The pupils, especially those in senior high school, likely confront numerous challenges. It is expected of these students to accurately comprehend what they have read. It helps them finish their reading comprehension assignments and broadens their perspective on current global challenges or the 4.0 Industrial Era. (Sanjaya & Hidayat, 2022) According to Al Hashimi et al., (2022) The 4.0 Industrial Revolution was an era where humans depended too much on technology which gradually changed human activities from conventional to digital. From communication, sharing thoughts (thinking), absorbing information to even reading. In the scale of absorbing information, the fast-changing and easy-to-get information where everything seemed to be very open and vast, media is releasing massive information on digital platforms where everyone can access it easily, but the problem is many of us cannot even recognize or tell whether it is a fact or opinion without reading it critically which is why many of us ended up believe in hoaxes. To avoid the negative of digital usage, it is important to understand the information as a whole by reading it very carefully and thoroughly, in other words, doing a reading comprehension. As a matter of fact, the Indonesian students’ reading performance from the reading test released by PISA in 2018 was ranked 74 of 80 countries which means Indonesian students’ reading skills are classified as very low and below average. This was a real challenge to teachers and lecturers, they had to find a way to improve or fix this matter. (Larking, 2017) According to G. Amirova, (2020) All pupils are currently coping with the 4.0 Industrial Revolution, which is different from the previous year. In educating kids and preparing them for that era, teachers are essential. In the 4.0 age, both people and technology will compete for an individual's attention. The preparation needs to be very good in order to boost the pupils' competence. One of the traits that might improve their lives is high-level thinking ability, often known as higher-order thinking Skills (HOTS). On the other hand, one of the numerous objectives of education in the twenty- Syah Ibi Hamma1, Syamsiarna Nappu2,Erwin Akib3 Assessing Students’ Higher-Order Thinking Skills in Reading Comprehension 1016 first century is to help students acquire High Order Thinking Skills, a combination of problem-solving and critical thinking skills. With the help of HOTS, students may better adapt to real-world contexts or scenarios, which helps them not only retain the information but also build the connections between knowledge and application so they can use it to make decisions in their daily lives. (Akib & Ghafar, 2015) There are three groups of assessment abilities that exist. High-level thinking abilities (HOTS) include the capacity to transfer one idea to another, the capacity to connect with people in unfamiliar situations, the capacity to think critically (critical thinking skills), the capacity to comprehend logical puzzles, the capacity to reason in a way that is focused on making a choice or acting, and the capacity to solve problems by coming up with novel solutions. (Brookhart, 2018). K13's goal is to foster students' critical thinking skills through a test that contains C4 questions that might elicit their ability to analyze, evaluate, and create, the assessment of HOTS is necessary. Numerous research looks into how to evaluate HOTS in the context of English as a Foreign Language. First, Johansson (2020) earlier research focused on how HOTS is evaluated in online EFL classes. The finding demonstrates that in order to produce outcomes that are both linguistically and cognitively advantageous while assessing HOTS, teachers must carefully plan and develop e-assessment assignments. It demonstrates that creating electronic assessment tasks to evaluate HOTS is feasible. Second, Pratiwi et al. (2019) conducted a study on how HOTS was reflected in summative assessments given to EFL teachers. Her research aims to examine how Higher-Order Thinking Skills (HOTS) are reflected in summative evaluations given by English teachers to students in the eleventh grade. According to the findings of her research, 5.4% of HOTS-based teacher-made exam items were still only available at the cognitive level of analysis (C4). According to the analysis's findings, the lower-order thinking skills (LOTS) category still applies to the cognitive levels of the children. Third, Amali (2018) sought to describe how teachers approach, use, and evaluate the HOTS evaluation when instructing reading comprehension. It used a sequential exploratory design. using the method of purposive sampling. According to the survey, teachers had a positive opinion of the concept and execution of the HOTS evaluation. Utilizing the proper instructional tactics and procedures for reading comprehension, they applied the HOTS assessment. In order to evaluate reading comprehension, the teachers also created the appropriate exam utilizing the HOTS assessment. The students' reading comprehension score was likewise at the necessary level. Despite the teachers' excellent performance, they encountered challenges nonetheless because of the wants and motivations of the students. Fourth, Turidho (2021) investigated instructors' knowledge of HOTS, their comprehension of the HOTS assessment, their assessment procedures, and the IDEAS, Vol. 11, No. 2, December 2023 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 1017 challenges they faced in senior secondary vocational schools. The results showed that practically all chemistry instructors thought giving HOTS to students in senior secondary vocational institutions was crucial. However, most teachers still lacked appropriate knowledge of HOTS, which made it difficult for them to assess students' competence. It was also discovered that misconceptions about higher-order thinking existed, such as the connection between higher-order thinking and item difficulty. The teachers in this study evaluated student characteristics, such as a lack of desire, poor academic performance, and poor reading habits, as hurdles to developing HOTS in chemistry instruction in vocational senior secondary schools. Fifth, Abkary & Purnawarman, (2020) explores the difficulties in assessing students' HOTS and the techniques used by EFL teachers. The study employed a qualitative approach, particularly a descriptive one. the information gathered through surveys and interviews about how teachers rate students' HOTS and its difficulties. According to the data, summative evaluation and authentic assessment are the two ways 22 EFL teachers most frequently employ to gauge their students' HOTS. Additionally, due to a lack of knowledge about HOTS assessment, the teachers never created HOTS questions. This suggested that teachers had difficulty evaluating students' HOTS. Teachers' and students' grasp of the HOTS assessment idea are the two areas where there are issues. Sixth, In order to close the existing gap and provide a model of higher-order thinking skills (HOTS)-based formative assessments for English learning, particularly in junior high schools, Wiyaka et al., (2020) conducted Higher- Order Thinking Skills (HOTS)-based Formative Assessment. This research outlines the validation of the assessment model. The suggested assessment methodology could serve as a model for measuring language learning. When teachers want to enhance the effectiveness of teaching and learning while also fostering students' critical thinking abilities, they can use this model of HOTS- based assessments as practical guidance. Seventh, Retnawati et al., (2018) conducted a study that aimed to describe instructors' understanding of higher-order thinking skills (HOTS). The study uses a qualitative phenomenological method of inquiry. The findings show that teachers' understanding of HOTS and their capacity to raise students' HOTS, address HOTS-related issues, and test students' HOTS are still lacking. However, it is true that teachers are already aware of the significance of HOTS and how to teach it utilizing a variety of cutting-edge teaching techniques. None of them conducted tests for high-order thinking in the areas of transfer, critical thinking, and problem-solving in reading comprehension. A preliminary study of this research was done at SMA Pesantren IMMIM Makassar where he did an observation on the twelfth grade and found that the students in grade 12 IPA 1 are the favorite one, they use English as a language to communicate with each other in daily life. As a matter of fact, Pesantren IMMIM Makassar has two languages to use every day in the school’s environment which Syah Ibi Hamma1, Syamsiarna Nappu2,Erwin Akib3 Assessing Students’ Higher-Order Thinking Skills in Reading Comprehension 1018 are English and Arabic. Based on the preliminary data above this research will assess the score of students’ HOTS and portray the challenges that students faced when solving HOTS which is why this research is going to conduct a research entitled Assessing Students’ HOTS in SMA Pesantren IMMIM Makassar at Twelfth Grade And the research questions are formulated as follows: 1) What are the scores of students’ Higher-Order Thinking Skills In Reading Comprehension in the Twelfth Grade of SMA Pesantren IMMIM Makassar? 2) What challenges are faced by the students when solving HOTS Questions? Literature Review Reading Assessment There are many uses for reading assessments, but they all start with a knowledge of the reading construct, awareness of how reading skills evolve, and an effort to reflect the construct in assessment tasks. For many teachers and managers, reading assessments can be intimidating and even overwhelming. As a result, the first objective of this chapter is to present a clear framework that classifies the various uses and purposes for assessment. For many teachers and managers, reading assessments can be intimidating and even overwhelming. As a result, the first objective of this chapter is to present a clear framework that classifies the various uses and purposes for assessment. A reasonably straightforward yet comprehensive framework should enable readers to sort through their own assessment experiences in a manner that lends the framework interpretive force. According to Nuttal (2000) in order for students to fully grasp a text, they need to be able to identify the main idea, locate specific information, draw conclusions, recognize references, and comprehend word meanings. These elements are viewed as challenges that the students face in understanding the book. Determining Main Idea A sentence that expresses the author's position on the subject is the primary idea. According to Cachia et al., (2018) the point of the passage, minus all the details, is the primary idea of a paragraph. Students are expected to identify the text's major idea when determining the main idea, so they should understand the subject matter of the text. The first sentence which is typically the primary idea can also be the middle or the final sentence. As a result, finding the main concept may become more challenging. The students might find it difficult to determine a passage's primary idea and where it is located. Making Inference Students are expected to understand the text in order to draw conclusions from the assertions in the text when drawing an inference. For instance, "What is the IDEAS, Vol. 11, No. 2, December 2023 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 1019 significance of the above statement?" This implies that the students must draw a conclusion from the remark. Kopitski (2007) added that in order to draw conclusions from the text's hints and prior information, readers must practice doing so. Because the statement's meaning is sometimes not stated in the text, it can be challenging for students to locate the text's end. Understanding the Meaning of Word In order to understand the meaning of a word, students must first understand the meaning of the phrase or text in which it appears, then they must determine the appropriate meaning of the word. Vocabulary is another crucial aspect of reading comprehension that language learners will struggle with, as noted by (Tajino et al., 2015). Therefore, before students read the material, it is necessary to teach them unfamiliar vocabulary related to the text so that they can easily comprehend the material. Goals for reading assessment Although it is essential to note that different assessment practices may assume different theories of reading and reading development, reading assessments are intended to provide feedback on the skills, processes, and knowledge resources that reflect reading abilities. All assessment frameworks serve significant purposes, and there are numerous methods to categorize assessment in general. Assessments are frequently divided into four categories: proficiency, achievement, placement, and diagnostic evaluations. These categories include norm-reference and criteria on reference testing, formative and summative evaluations, formal and informal (or alternative) evaluations, and norm-reference and criteria on reference testing. For the goals of this book, there are five fundamental assessment purposes that are used to organize and describe the reading assessment process. There are five goals for literacy evaluations: Higher-Order Thinking Skills Higher-order thinking skills are a series of important competencies individuals can utilize in order to improve learning progress and critical thinking. Those who employ high-order thinking skills understand how to analyze and evaluate complex information, categorize, manipulate, and connect facts, troubleshoot for solutions, understand concepts, connections, and big-picture thinking, problem solve, ideate, and develop insightful reasoning. High-order thinking skills, also called high-order thinking or HOT, refer to skills that go beyond memorizing information or regurgitating stories skills at the bottom of Bloom’s Taxonomy hierarchy and emphasize the development of analytical skills. High-order thinking skills are thought to be harder to teach and learn than mere facts but are ultimately more important for developing critical thinking and analytical faculties. Though essential at the university and college level, high-order thinking skills are thought to be increasingly important in early education. Syah Ibi Hamma1, Syamsiarna Nappu2,Erwin Akib3 Assessing Students’ Higher-Order Thinking Skills in Reading Comprehension 1020 Research Method This study used a quantitative research methodology. According to Creswell (2018) quantitative research is a technique for examining the relationship between variables to evaluate objective hypotheses. These variables can be measured, frequently with the aid of instruments, to allow statistical analysis of numbered data. This research used Higher-Order Thinking Skills questions as an instrument to determine and analyze the score of students' HOTS in Critical Reading Comprehension Population and Sample This research utilized a descriptive research design using quantitative data. The sample is 22 students from SMA Pesantren IMMIM Makassar. The students learn the English Language and they are selected using a total sampling technique Research Instrument Test This research used the HOTS-Based Reading Literacy Assessment Instrument by giving learners’ questions as instruments to collect data or gather information from learners. A HOTS-Based Reading Literacy Assessment Instrument was used to collect the data about students’ HOTS scores, received from the test. Questionnaire This research used a questionnaire to investigate the student’s challenges when solving HOTS in Reading Comprehension questions. Data Collection In collecting data the writer applies the procedures as follows: HOTS-Based Reading Literacy Assessment This research had the permission of SMA Pesantren IMMIM English Teacher for assessing students’ HOTS scores. The writer makes a rubric score about students' HOTS score, which consists of three main scoring: Higher-Order Thinking as Transfer, Higher-Order Thinking as Critical Thinking, and Higher-Order Thinking as Problem Solving. Data Analysis The data in this research was analyzed through SPSS Version 25. Below were several data that were calculated: Analyzed the percentage of students’ HOTS Calculated the mean of the students’ HOTS score Table 3.1 Category for assessing higher-order thinking skills Score Interval Level of students’ higher order thinking 74-100 Excellent IDEAS, Vol. 11, No. 2, December 2023 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 1021 47-73 Good 0-46 Poor Adapted from Akib & Ghafar, (2015) Table 3.2 Scoring Guidance and The explanation of Criterion: Scoring rubric of HOTS in Reading Comprehension Item Analysis Score Criterion of Scoring High Order Thinking as Transfer 20 10 0 High : Evidence is clear, accurate and complete Mid. : Evidence is mostly clear, relevant and complete Low : Evidence is not clear, relevant and complete High Order Thinking as Critical Thinking 20 10 0 High : Evidence is clear, accurate and complete Mid. : Evidence is mostly clear, relevant and complete Low : Evidence is not clear, relevant and complete High Order Thinking as Problem Solving 41-60 21-40 0-20 High : Clear, appropriate identification of two Additional pieces of information. Mid : Appropriateness of evidence Low : Soundness of reasoning and clarity of explanation Total 100 Adopted from Brookhart (2011) Table 3.3 Interval Students’ Challenge When Solving HOTS In Reading Comprehension Criteria Criteria Percentage Strongly Agree 84% < Score <100% Agree 68% < Score <83% Neutral 52%< Score <67% Syah Ibi Hamma1, Syamsiarna Nappu2,Erwin Akib3 Assessing Students’ Higher-Order Thinking Skills in Reading Comprehension 1022 Disagree 36%< Score <52% Strongly Disagree 35%< Adopted from A. Al Roomy, (2022) Table 3.4 Scoring Guidance and the explanation of Criterion: Scoring Questionnaire rubric No. Indicators Aspect of Challenges Layout SD D N A SA 1 I cannot remember vocabulary technically Understanding the meaning of word 2 I cannot deduce the information from the questions. 3 I cannot determine the relevant information in questions. Making inference 4 I cannot classified detailed information in questions. 5 I cannot state the meaning of the terms that represents the questions. Determining the main idea 6 I cannot remember one or more necessary conditions for an object to be expressed in the term that represent the questions. Adapted from Amalia Rakhmyta & Maulidiyah (2021) Findings and Discussions The Score of Students Higher Order Thinking Skills In Reading Comprehension In this research, students’ Higher-Order Thinking Skills in Reading IDEAS, Vol. 11, No. 2, December 2023 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 1023 Comprehension were analyzed and calculated quantitatively by using a test that included 6 questions which were 4 multiple choices and 2 open-ended questions. The multiple-choice question scored 10 per each if correct and 0 if incorrect, and the open-ended questions scored with 0-30 each. In this section, the students’ HOTS score was presented as follow: Table 4.1 The Score of Students’ HOTS in Reading Comprehension Score Interval Amount of Students Level of students’ higher order thinking 74-100 8 Excellent 47-73 11 Good 10-46 3 Poor Table 4.1 shows the score of HOTS in Reading Comprehension answered by the students. The subjects of the test were students in Twelfth Grade IPA 1 of SMA Pesantren IMMIM Makassar. There were 22 students that tested which 2 students scored 40, 2 students scored 50, 7 students scored 60, 2 students scored 70, 6 students scored 80, and a student scored 90, with the highest score was 100 (perfect score) while the lowest score was 10 and the average score was 64.54. Table 4.2 The Category of Students’ HOTS in Reading Comprehension Frequency Percent Valid Percent Cumulative Percent Poor 3 13.6 13.6 13.6 Good 11 50.0 50.0 63.6 Excellent 8 36.4 36.4 100.0 Total 22 100.0 100.0 Table 4.2 shows the category of students HOTS answered by the students. The data from the previous table was analyzed through SPSS to find which category of students’ HOTS in Reading Comprehension and the results showed there were 3 students scored below average or (13.6%) in percentage which was categorized as Poor category, there were 11 students scored average or (50%) in percentage which was categorized as Good category and 8 students scored above average or (36.4%) in percentage which was categorized as Excellent category. Table 4.2.1 Mean Scores of Students’ Higher Order Thinking Syah Ibi Hamma1, Syamsiarna Nappu2,Erwin Akib3 Assessing Students’ Higher-Order Thinking Skills in Reading Comprehension 1024 Types of Higher Order Thinking N Mean Std. Deviation Maximum Higher-Order Thinking as Transfer (C4) 22 18.1818 3.94771 20 Higher-Order Thinking as Critical Thinking (C5) 21 17.6190 5.38958 20 Higher-Order Thinking as Problem solving (C6) 22 28.6364 15.82507 60 Table 4.2.1 shows the mean score of students in Higher-Order Thinking as Transfer (C4) has the highest mean (Mean=18.1818; SD=3.94771) than the Higher-Order Thinking as Critical Thinking (C4) (Mean=17.6190; SD=5.38958) and High Order Thinking as Problem Solving has the lowest compare to the maximum score that given (Mean=28.6364; SD;15.82507) Table 4.3 The Score of Students’ In High-Order Thinking as Transfer Questions Frequency Percent Valid Percent Cumulative Percent Poor 0 0 0 0 Good 4 18.2 18.2 18.2 Excellent 18 81.8 81.8 100.0 Total 22 100.0 100.0 Table 4.3 shows the score of students in Higher-Order Thinking as Transfer that was answered by 22 students. The results show there were no students who scored below average or (0%) in percentage which was categorized as the Poor category, there were 4 students who scored average or (18.2%) in percentage which was categorized as the Good category and 18 students scored above average or (81.8%) in percentage which was categorized as Excellent category. There were 2 multiple-choice questions that indicated Higher-Order Thinking as Transfer which were answered by the students. Higher-order thinking as transfer indicates meaningful learning. While retention requires students to remember what they have learned, transfer requires students to make sense of and be able to use what they have learned. IDEAS, Vol. 11, No. 2, December 2023 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 1025 Table 4.4 The Score of Students’ In Higher-Order Thinking as Critical Thinking Questions Frequency Percent Valid Percent Cumulative Percent Poor 2 9.1 9.1 9.1 Good 8 46.7 46.7 45.5 Excellent 12 54.5 54.5 100.0 Total 22 100.0 100.0 Table 4.4 shows the score of students in Higher-Order Thinking as Critical Thinking which was answered by 22 students. the results show there were 2 students who scored below average or (9.1%) in percentage which was categorized as the Poor category, there were 8 students who scored average or (46.7%) in percentage which was categorized as the Good category and 12 students scored above average or (54.5%) in percentage which was categorized as Excellent category. There are two multiple-choice questions that indicate Higher-Order Thinking as Critical Thinking which were answered by the students. Higher-Order Thinking as Critical Thinking is reasonable, reflective thinking that is focused on deciding what to believe or do which includes reasoning, questioning and investigating, observing and describing, comparing and connecting, finding complexity, and exploring viewpoints. Table 4.5 The Score of Students’ In Higher-Order Thinking as Problem Solving Questions Frequency Percent Valid Percent Cumulative Percent Poor 12 54.5 54.5 54.5 Good 8 36.4 36.4 90.9 Excellent 2 9.1 9.1 100.0 Total 22 100.0 100.0 Table 4.5 shows the score of students in Higher-Order Thinking as Problem- Solving that was answered by 22 students. the results show there were 12 students scored below average or (54.5%) in percentage which was categorized as the Poor category, there were 8 students scored average or (36.4%) in percentage which was Syah Ibi Hamma1, Syamsiarna Nappu2,Erwin Akib3 Assessing Students’ Higher-Order Thinking Skills in Reading Comprehension 1026 categorized as the Good category and 2 students scored above average or (9.1%) in percentage which was categorized as Excellent category There are two open-ended questions that indicate Higher-Order Thinking as Problem-Solving which were answered by the students. Higher-order thinking as problem-solving when a student incurs a problem then the student wants to reach a specific outcome or goal but does not automatically recognize the proper path or solution to use to reach it. The problem to solve is how to reach the desired goal but many problems are open-ended, could have many correct solutions or multiple paths to the same solution, or are genuine questions for which answers are not known. The Challenges Faced by the Students When Solving Higher-Order Thinking Skils In Reading Comprehension The challenges that students face when solving Higher-Order Thinking Skills Questions in Reading Comprehension were analyzed and calculated quantitatively by using SPSS from the questionnaire that included 6 questions which were 2 questions for Understanding the Meaning of Word’s challenge category, 2 questions for Making Inference’s challenge category and 2 questions for Determining Main Idea’s challenge category. The questions will be scored 1-5 per each, The challenges faced by the students when solving HOTS in Reading Comprehension score were presented as follows: Table 4.6 Challenges in Understanding the Meaning of Word No. Statements Aspect of Challenge Layout SD D N A SA 1 I cannot remember vocabulary technically Understan ding the meaning of word 2 (9.0 9%) 14 (63. 63% ) 2 (9.0 9%) 2 (9. 09 %) 2 (9.0 9%) 2 I cannot deduce the information from the questions. 4 (18. 18% ) 10 (45. 45% ) 4 (18. 18% ) 2 (9. 09 %) 2 (9.0 9%) Table 4.6 shows the statements from students that indicate Challenges in Understanding the meaning of Words when Solving Higher-Order Thinking questions in Reading Comprehension, on the Strongly Agree (SA) the first statement comprised 2 (9.09%) students, the second statement comprised 2 (9.09%) student. On the Agree Scale (A) the first statement comprised 2 (9.09%) students, and the second statement comprised 2 (9.09%) students. On the Neutral Scale (N) the first IDEAS, Vol. 11, No. 2, December 2023 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 1027 statement comprised 2 (9%) students, and the second statement comprised 4 (18%) students. On the Disagree Scale (D) the first statement comprised 18 (82%) students. The second statement comprised of 14 (64%) students. On the Strongly Agree Scale (SD) the first statement comprised 2 (9%) students, and the second statement comprised 4 (18%) students. The total score was 108 then divided into 660 (Total Score in aspect of challenge in Making Inference) 0.16363 then multiplied by 100 then the result is 23.63%. there is 16.63% students that find it difficult in understanding the meaning of word when solving HOTS questions. Table 4.7 Challenges in Making Inferences No. Statements Aspect of Challenge Layout SD D N A SA 1 I cannot determine the relevant information in questions. Making Inference 1 (4.5 4%) 1 (4.5 4%) 1 (4.5 4%) 11 (50 %) 8 (36. 36% ) 2 I cannot classified detailed information in questions. 1 (4.5 4%) 1 (4.5 4%) 2 (9.0 9%) 6 (27 .27 %) 12 (54. 54% ) Table 4.7 shows the statement from students that indicate Challenges in Making Inferences when Solving Higher-Order Thinking questions in Reading Comprehension, on the Strongly Agree (SA) the first statement comprised 8 (36.36%) students, the second statement comprised 12 (54.54%) students. On the Agree Scale (A) the first statement comprised 11 (50%) students, and the second statement comprised 6 (27.27%) students. On the Neutral Scale (N) the first statement comprised of 1 (4.54%) student, and the second statement comprised of 2 (9.09%) students. On the Disagree Scale (D) the first statement comprised of 1 (4.54%) student. The second statement comprised of 1 (4.54%) student. On the Strongly Agree Scale (SD) the first statement comprised of 1 (4.54%) students, and the second statement comprised of 4 (4.54%) students. The total score was 183 then divided into 660 (Total Score in the aspect of challenges in Making Inference 0.2772 then multiplied by 100. The result is 27.72%. there is 27.72% of students find it difficult to make inferences when solving HOTS questions. Table 4.8 Syah Ibi Hamma1, Syamsiarna Nappu2,Erwin Akib3 Assessing Students’ Higher-Order Thinking Skills in Reading Comprehension 1028 Challenges in Determining the Main Idea No. Statements Aspect of Challenge Layout SD D N A SA 1 I cannot state the meaning of the terms that represents the questions. Determinin g the main Idea 2 (9. 09 %) 1 (4.5 4%) 1 (4.5 4%) 11 (50 %) 7 (31. 81% ) 2 I cannot remember one or more necessary conditions for an object to be expressed in the term that represent the questions. 1 (4. 54 %) 2 (9.0 9%) 2 (9.0 9%) 8 (36 .36 %) 9 (40. 90% ) Table 4.7 shows the answers from students that indicate Challenges in Determining the main idea when Solving Higher-Order Thinking questions in Reading Comprehension, on the Strongly Agree (SA) the first statement comprised 7 (31.81%) students, the second statement comprised 9 (40.09%) students. On the Agree Scale (A) the first statement comprised 11 (50%) students, and the second statement comprised 8 (36.36%) students. On the Neutral Scale (N) the first statement comprised of 1 (4.54%) student, and the second statement comprised of 2 (9.09%) students. On the Disagree Scale (D) the first statement comprised of 1 (4.54%) student. The second statement comprised of 2 (9.09%) students. On the Strongly Agree Scale (SD) the first statement comprised 2 (9.09%) students, and the second statement comprised 4 (4.54%) students. The total score was 186 then divided into 660 (Total Score in the aspect of challenges in Making Inference) 0.2818 then multiplied by 100. The result is 28.18%. there is 28.81% of students find it difficult to determine the main idea when solving HOTS questions. Conclusions Based on the analysis of the test, it can be seen that the score of students’ HOTS In Reading Comprehension was good, which were the score of Higher-Order Thinking as Transfer (C4) was Excellent, Higher-Order Thinking as Critical Thinking (C5) was Good and Higher-Order Thinking as Problem Solving was Poor and it could be concluded that the result of students HOTS In Reading Comprehension test was Excellent in Higher-Order Thinking as Transfer (C4), Good in Higher-Order Thinking as Critical Thinking (C5) and Poor in Higher-Order Thinking as Problem Solving (C6) In terms to the challenges, it can be seen that the challenges that students face when solving HOTS in Reading Comprehension were 16.63% students facing difficulties in IDEAS, Vol. 11, No. 2, December 2023 ISSN 2338-4778 (Print) ISSN 2548-4192 (Online) 1029 understanding the meaning of words (C4), 27.72% students facing difficulties in making inferences (C5) and 28.81% students facing difficulties in determining the main idea (C6) References A. Al Roomy, M. (2022). Investigating the Effects of Critical Reading Skills on Students’ Reading Comprehension. Arab World English Journal, 13(1). https://doi.org/10.24093/awej/vol13no1.24 Abkary, N. S., & Purnawarman, P. (2020). Indonesian EFL Teachers’ Challenges in Assessing Students’ Higher-Order Thinking Skills (HOTS). https://doi.org/10.2991/assehr.k.201215.076 Akib, E., & Ghafar, M. N. A. (2015). Assessment for learning instrumentation in higher education. International Education Studies, 8(4). https://doi.org/10.5539/ies.v8n4p166 Al Hashimi, S., Alamarat, Y., & Zaki, Y. (2022). Students’ perceptions of online assessment, feedback practices, and challenges. International Journal of Evaluation and Research in Education (IJERE), 11(4), 1939. https://doi.org/10.11591/ijere.v11i4.22753 Amalia Rakhmyta, Y., & Maulidiyah, A. (2021). STUDENTS’ DIFFICULTIES IN SOLVING HIGHER-ORDER THINKING SKILLS (HOTS) READING COMPREHENSION SECTION AT SMA 1 TAKENGON. In Jurnal As-Salam (Vol. 5, Issue 2). Brookhart, S. M. (2010). How to Assess Higher-Order Thinking Skills in Your Classroom. In Ascd. Brookhart, S. M. (2011). Educational Assessment Knowledge and Skills for Teachers. Educational Measurement: Issues and Practice, 30(1). https://doi.org/10.1111/j.1745-3992.2010.00195.x Cachia, A., Roell, M., Mangin, J. F., Sun, Z. Y., Jobert, A., Braga, L., Houde, O., Dehaene, S., & Borst, G. (2018). How interindividual differences in brain anatomy shape reading accuracy. Brain Structure and Function, 223(2). https://doi.org/10.1007/s00429- 017-1516-x Creswell, J. W., & Creswell, J. D. (2018). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches - John W. Creswell, J. David Creswell - Google Books. In SAGE Publications, Inc. G. Amirova. (2020). The benefits of reading skills in the educational process. Proceedings of The ICECRS, 3. https://doi.org/10.21070/icecrs2019463 Johansson, E. (2020). The Assessment of Higher-order Thinking Skills in Online EFL Courses: A Quantitative Content Analysis. NJES Nordic Journal of English Studies, 19(1). https://doi.org/10.35360/njes.519 Kopitski, M. (2007). Exploring the Teaching of Inference Skills. Newspaper Research Journal, 27(3). Larking, M. (2017). Critical Reading Strategies in the Advanced English Classroom. APU Journal of English Research, 2. Nurul Amali, L., Anggani Linggar Bharati, D., & Rozi, F. (n.d.). The Implementation of High Order Thinking Skills (HOTS) Assessment to Evaluate the Students’ Reading Syah Ibi Hamma1, Syamsiarna Nappu2,Erwin Akib3 Assessing Students’ Higher-Order Thinking Skills in Reading Comprehension 1030 Comprehension Achievement. http://journal.unnes.ac.id/sju/index.php/eej Nuttal, C. (2000). Teaching reading skills in a foreign language. In System. Pratiwi, N. P. W., Dewi, N. L. P. E. S., & Paramartha, A. A. G. Y. (2019). The Reflection of HOTS in EFL Teachers’ Summative Assessment. Journal of Education Research and Evaluation, 3(3). https://doi.org/10.23887/jere.v3i3.21853 Retnawati, H., Djidu, H., Kartianom, Apino, E., & Anazifa, R. D. (2018). Teachers’ knowledge about higher-order thinking skills and its learning strategy. Problems of Education in the 21st Century, 76(2). https://doi.org/10.33225/pec/18.76.215 Sanjaya, B., & Hidayat, W. (2022). Student speaking skill assessment: Techniques and results. International Journal of Evaluation and Research in Education (IJERE), 11(4), 1741. https://doi.org/10.11591/ijere.v11i4.22782 Tajino, A., Stewart, T., & Dalsky, D. (2015). Team Teaching and Team Learning in the Language Classroom. In Team Teaching and Team Learning in the Language Classroom. https://doi.org/10.4324/9781315718507 Turidho, A., Oktalidiasari, D., & Wahyu Saputri, N. (n.d.). Reading Assessment: Higher- Order Thinking Skills (Hots) through ICT Graduate Students of Mathematics Education, Sriwijaya University (Vol. 20, Issue 1). Wiyaka, W., Prastikawati, E. F., & Kusumo Adi, A. P. (2020). Higher-Order Thinking Skills (HOTS)-based Formative Assessment: A Proposed Model for Language Learning Assessment. Vision: Journal for Language and Foreign Language Learning, 9(2). https://doi.org/10.21580/vjv9i25859 Alfatihah, A., Ismayanti, D., Syam, A., & Santaria, R. (2022). Teaching Speaking Skills through Project-Based Learning for the Eighth Graders of Junior High School. IDEAS: Journal on English Language Teaching and Learning, Linguistics and Literature, 10(1), 152-165. doi:https://doi.org/10.24256/ideas.v10i1.2555 Sari, E., & Agustina, M. (2022). Thematic Development in Students’ Argumentative Essay. IDEAS: Journal on English Language Teaching and Learning, Linguistics and Literature, 10(1), 166-174. doi:https://doi.org/10.24256/ideas.v10i1.2563 Rahma, Q. (2022). Developing Multimodal Mindful Material for Secondary School in Indonesia Post Covid-19 Pandemic. IDEAS: Journal on English Language Teaching and Learning, Linguistics and Literature, 10(2), 1220-1234. doi:https://doi.org/10.24256/ideas.v10i2.3227 https://doi.org/10.21580/vjv9i25859 https://doi.org/10.24256/ideas.v10i1.2555 https://doi.org/10.24256/ideas.v10i1.2563 https://doi.org/10.24256/ideas.v10i2.3227