32 JISAE. Volume 3 Number 1 February 2017. Copyright © Ikacana Publisher | ISSN: 2442-4919 HIGHER ORDER THINKING SKILLS ASSESSMENT (HOTS) I Wayan Widana IKIP PGRI Bali, Denpasar Email: i.wayan.widana.bali@gmail.com ABSTRACT This study aims to: 1) provide knowledge and understanding to the teachers about the concept and characteristics of the HOTS (Higher Order Thinking Skills) assessment extensively and deep; and 2) improve the skills of the teachers to develop HOTS assessment. To achieve Reviews These objectives, the research literature sourced from journals, reference books, modules, Internet, and other sources such as the print and electronic media that are relevant to the topic HOTS development assessment. Data collected by documentation and observation techniques. The research of data were Analyzed using descriptive exploratory methods. The results of the data analysis are as follows: 1) HOTS assessments are questions or tasks that have Reviews These characteristics: to assess students abilities to analyze, Evaluate, and create based on contextual issues, and are not routine (not familiar); 2) steps compose HOTS items are: a) analyze the KD that can be created HOTS items, b) arrange the blueprint of HOTS items, c) write down the items on the card matter, d) Determine the answer key (multiple choice questions form) or arrange rubric / scoring guidelines (essay form), e) perform qualitative analysis, and f) perform quantitative analysis; 3) the advantage of HOTS assessments are: a) Increase the stundents motivation to learn and b) improve learning outcomes. and f) perform quantitative analysis; 3) the advantage of HOTS assessments are: a) Increase the stundents motivation to learn and b) improve learning outcomes. and f) perform quantitative analysis; 3) the advantage of HOTS assessments are: a) Increase the stundents motivation to learn and b) improve learning outcomes. Keywords: assessment, higher-order thinking skills. Nizam (2016) stated that the assessment in Indonesia directed at the assessment model of Higher Order Thinking Skills (HOTS). The policy refers to the need for life skills in the 21st century. Bernie Trilling (2005) formulated the life skills of the 21st century in the form of The Seven C's 21st Century Lifelong Skills: (1) critical thinking, (2) creativity, (3) communication, (4) collaboration, (5) career and learning self-reliance, (6) cross- cultural understanding, and (7) computing / ICT literacy. Joke Voogt & Natalie Pareja Roblin (2010) suggested that the assessment HOTS, can enhance critical thinking skills (critical thinking), creativity (creativity) and confidence (learning self-reliance). The implication of the above policy is the teachers are encouraged to develop assessment instruments HOTS, both in daily tests, a final assessment of the semester, and school examinations (US). Teachers can be creative forms of matter in accordance with the Basic Competency (KD) in the respective subjects, and raised the contextual issues that exist in the school environment as stimulus material HOTS assessment. It is intended to show that the material taught in class was not impressed merely theoretical only, but can also be used to solve problems in the contextual environment of the learners so as to motivate the students to study harder. The results of the qualitative analysis item Leads US in 614 SMA (superior school) in Indonesia, which is conducted by the Directorate of School (2016) in Quality Improvement activities Penyenggaraan US data showed US cognitive problems as follows. 33 Table 1. Cognitive realm About US Cognitive realm many Item % Knowing (C1) 2,917 19.55 Understanding (C2) 7064 47.35 Applying (C3) 3,807 25.52 Analyzing (C4) 1,015 6,80 Evaluating (C5) 99 0.66 Mengkreasi (C6) 17 0.11 amount 14 919 100 Only about 7.58% of US-made problem of high school teachers who belong to the realm Reference C4, C5, and C6. Problems US compiled by teachers generally measures the skills of remembering and understanding (knowing and understanding). When viewed from the context of most use in the classroom context and highly theoretical, and rarely use outside the classroom context (contextual). So it does not show the link between knowledge acquired in learning with real situations in everyday life. This can give you an idea that the understanding of the concept of HOTS assessment and the ability of teachers to prepare about HOTS is still very low, it has implications for the learners are not accustomed to working on the problems HOTS. One of the effects of the low ability of teachers write the questions HOTS is the low achievement by learners in Indonesia in a study conducted by the International Institute PISA (Program for International Students Assessment). PISA measures three basic literacy, namely: scientific literacy, reading, and mathematics. In 2015, for scientific literacy learners Indonesia only able to attain the rank 62nd out of 70 countries. As for the reading and mathematical literacy successive ranks 64th and 63rd out of 70 countries (PISA, 2016). This shows that the competitiveness of Indonesia at the international level is very low. Therefore, teachers need to be motivated and given extensive information about the assessment HOTS, so skilled and creative composing about HOTS. One study on the development of HOTS questions on subjects Physics in high school has been conducted by Edi Istiyono, et al (2014), entitled Development of Higher- Order Thinking Ability Test Physics (PysTHOTS) High School Students. The instrument consists of two sets of tests, each of which has 26 items including eight anchor items and has been validated by measurement experts, expert physics education, physicists, and practitioners. The instrument has been validated tested on 1,001 students from ten SMAN in Yogyakarta. Politomus Data were analyzed using the Partial Credit Model (PCM). The trial results showed that all the items were 44 and instrument PhysTHOTS proven fit with PCM, instrument reliability of 0.95, the index of difficulty of items ranging -0.86 to 1.06, which means all items in both categories. Research conducted by Edi Istiyono, et al (2014), specifically made in physics in high school. While in this study, emphasizes the study of the development about HOTS expanded at all levels of education from elementary, junior high, high school / vocational school for all subjects. The goal is to eliminate the idea that the problems HOTS connoted as difficult problems in secondary education and above, and can only be developed on specific subjects such as Mathematics and Science. All teachers of subjects at all levels of education is expected to prepare HOTS assessment in accordance with the characteristics 34 JISAE. Volume 3 Number 1 February 2017. Copyright © Ikacana Publisher | ISSN: 2442-4919 of learners and the demands of the relevant KD. Therefore, teachers need to have a reference that is adequate and sufficient to build knowledge and insight into the assessment HOTS. Based on the above, it can be served formulation of the problem as follows: (1) what is a matter of HOTS ?; (2) how the characteristics of HOTS matter ?; (3) how to develop measures and guidelines HOTS items penskorannya? (4) what is the benefit assessment HOTS? In accordance with the formulation of the above problems, the study aims to: 1) provide knowledge and understanding to teachers about the concept and characteristics of HOTS assessment is broad and deep; and 2) improve the skills of teachers to prepare the assessment HOTS. Through their understanding and knowledge of the issues HOTS good, the teachers are expected to have adequate reference so that it can tell which assessment including HOTS HOTS or not. Because in practice, teachers are often confused or undecided whether the items are arranged already includes HOTS or not. In the end, knowledge and understanding are expected to improve the skills of teachers prepare (create) problems HOTS. Thus, gradually the quality of teacher assessment instruments used for daily tests, semester and US assessment for all subjects can be improved. METHODS This research includes literature research is research to various literature obtained from journals, reference books, modules, Internet, and other sources such as print and electronic media that are relevant to the topic HOTS assessment. The study was conducted from August 22 till 30 November 2016. The study begins by collecting a variety of print and electronic literature relating to the assessment HOTS. Data collected by documentation and observation techniques. The research data were analyzed using descriptive exploratory method. RESULTS Cognitive Taxonomy A discussion of the cognitive taxonomy is a bridge that can deliver a way of thinking to understand the concept and the characteristics of the assessment of Higher Order Thinking Skills (HOTS). There are several cognitive taxonomy. Taxonomy is the most popular cognitive Bloom's taxonomy. The main difference between the original and Taxonomy Bloom's Taxonomy Bloom's revised by Anderson & Krathwohl (2001) is the revised Bloom's Taxonomy by Anderson & Krathwohl (2001) consists of two dimensions of the dimension of knowledge and cognitive processes. The Knowledge Dimension classify types of knowledge acquired learners into four kinds, namely: (a) the factual knowledge includes knowledge of symbols, symbols, notation, character names, historical events; (B) including the understanding of conceptual knowledge, definitions, theories, models, formulas; (C) the procedural knowledge relating to how, techniques, procedures, algorithms, steps to do something; and (d) metacognitive knowledge is a person's ability to understand and skillfully use the three-dimensional prior knowledge. While the dimensions of the cognitive process consists of six levels: (1) recall (remembering-C1) includes the ability to remember factual and conceptual knowledge; (2) understand (understanding-C2) 35 a basic understanding capabilities, build their own meaning; (3) apply (aplying-C3) means the ability to execute or implement procedures to solve the problem, it is generally an application that has the best answer; (4) analysis (analyzing-C4) means the ability to break down information into parts that are more specific, to find a link parts information with one another, and overall information, and includes the ability to distinguish, organizing, and connecting; (5) evaluate (Evaluating-C5) include the ability to judge something (justification) criteria, involves the ability to examine and scrutinize; (6) mengkreasi (creating-C6) means putting different elements together to form a new unity or reorganization of existing elements to form a new structure, including the process of generating, planning, and producing. Assessment standards for the Dimensions of Learning models(Marzano, Pickering, & McTighe, 1993) distinguishes dimensions of knowledge into declarative knowledge, procedural, complex thinking, processing information, effective communication, cooperation, and habits of thought. Included in the assessment domain HOTS is a dimension of complex thinking, processing information, effective communication, cooperation, and habits of thought. Marzano and Kendall (2007) identified three dimensions of knowledge, namely: information, mental procedures, and psychomotor procedures. While the dimensions of the thought process comprising: (1) retrieval, (2) comprehension, (3) analysis, (4) knowledge utilization, (5) metacognition, and (6) self- system thinking. Furthermore, Webb (2002) classifies four levels of thinking required to perform various cognitive activities, namely: (1) recall and reproduction, (2) the skill and concept, (3) strategic thinking, and (4) extended thinking. Biggs and Collis (1982) suggested taxonomy SOLO (Structure of Observed Learning Outcomes), a hierarchical taxonomy of thinking skills that consists of five levels: (1) prestructural, (2) unistructural, (3) multistructural, (4) relational, and (5) extended abstract. From the above, it appears that each of the cognitive taxonomy there are similarities in formulating high-level thinking skills (HOTS). Learners are required to have the ability to think more complex, the ability to connect the fragments of information into a single entity, the ability to transfer more complex concept in a new situation. Higher Order Thinking Skills Assessment (HOTS) King, FJ, Ludwika Goodson., & Faranak R. (2012) defines a high-level thinking skills (HOTS) as a critical thinking skills, logical, reflective, metacognitive, and creative. These capabilities will develop when people have problems that are not familiar, uncertainties, or a new phenomenon that requires solutions that have never been done before. Assessment is an assessment that measures HOTS third highest level in Bloom's Taxonomy to analyze, evaluate and create (Anderson & Krathwohl, 2001; Robert H. Ennis, 2014; Swartz and McGuinness, 2014). While Arter, Judith A, Salmon, & Jennifer R (1987) states that some capabilities that are included in the assessment HOTS are: (1) problem solving (problem solving ability), (2) decision making (decision-making), Susan M. Brookhart (2010) defines three categories of assessment capabilities HOTS as follows: (1) the ability to transfer the concept to another concept, a high-level thinking skills as a form of knowledge possessed the ability to connect with others who are not familiar situations; (2) critical thinking skills (critical thinking skills), is the ability to understand the problems of logic, reflective thinking skills, the ability to argue that can be focused to take a decision or do something; and (3) the ability of problem solving (problem 36 JISAE. Volume 3 Number 1 February 2017. Copyright © Ikacana Publisher | ISSN: 2442-4919 solving), namely the ability to find a new way, a solution that is not common, defining the problem creatively. Furthermore Widana, I Wayan (2016) states that the assessment HOTS is a measurement instrument used to measure the ability to think critically, the ability to think that not only recall (recall), restate (restate), or refer without processing (Recite) , Assessment HOTS measure the ability to: 1) transfer the concept to another concept, 2) process and apply information, 3) looking for connection of a variety of information that is different, 4) use information to solve problems, and 5) examines the ideas and information critical , Judging from the dimensions of knowledge, generally assess HOTS measure metacognitive dimensions, not merely measure the dimensions of the factual, conceptual or procedural. Metacognitive dimension describes the ability to connect several different concepts, interpret, The Australian Council for Educational Research(ACER, 2015) states that creativity to solve the problems in HOTS, consisting of: a) the ability to resolve problems that are not familiar; b) the ability to evaluate the strategies used to solve the problem from different angles; and c) finding new models of settlement of different ways before. The difficulty level is not the same item with a high level thinking skills. For example, to determine the meaning of a word that is not common (uncommon word) may have a very high degree of difficulty, but the ability to answer these problems do not include higher-order thinking skills. Thus, the assessment HOTS is not always the questions that have a high level of difficulty. High-level thinking skills can be trained in the learning process in the classroom. Therefore, to make the students have the ability to think critically, then the learning process also provides a space for learners to find the concept of knowledge-based activities. Activities in learning to encourage students to develop creativity and critical thinking. Characteristics Assessment HOTS Based on the above notions can be formulated characteristics HOTS assessment is as follows. 1) Measuring the ability of a high level. High-level thinking skills, including the ability to solve problems (problem solving), critical thinking skills (critical thinking), creative thinking (creative thinking), argued ability (reasoning), and the ability to make decisions (decision making). In Bloom's Taxonomy requires the ability to analyze (C4), evaluating (C5), and create (C6). 2) Based on contextual issues. Assessment HOTS typically loads stimulus in the form of case (based on case). Stimulus could lead learners to connect knowledge in different situations (the ability to transfer concepts). Case can be removed from the real situation in everyday life (contextual), such as global issues such as the issue of information technology, science, economics, health, education, character, and infrastructure. Here is outlined five characteristics contextual assessment, which is abbreviated REACT: a) Relating, the assessment is directly related to the context of real-life experience; b) Experiencing, assessment emphasized the excavation (exploration), discovery (discovery), and creation (creation); c) Applying, assessment demands the ability of learners to apply knowledge gained in the classroom to solve real problems; d) Communicating, assessment demands the ability of learners to be able to communicate conclusions models at the conclusion context of the problem; and e) 37 Transferring, assessment of learners who require the ability to transform the concepts of knowledge in the classroom into a new situation or context. 3) Not routine (not familiar). Assessment HOTS not regular assessments given in class. HOTS assessment used many times on the same test participant an assessment of memory (recall), because it has never been done before. HOTS unfamiliar assessment requires learners really think creatively, because of the problems encountered have never encountered or done before. The ability to analyze To assess the ability of learners to analyze a particular problem, question or a given task should measure the ability to locate specific parts or describe the parts are related to each other. Level analyze questions demonstrate the ability of learners to present concepts or issues that demonstrate the ability to compare parts with one another using logical arguments. Explaining the arguments used to connect the parts with one another. The types of questions that can be given is to analyze: 1) focus on the question or the main idea, generally finding the main idea of a paragraph that does not express explicitly or infer the main idea in the text as a whole; 2) analyze the arguments or conclusions, questions that could be asked, for example: what evidence is given to support the argument ?, author of what is given evidence that contrary to the argument writers ?, what assumptions required for the argument to be valid ?, is there a part of the statement not relevant to the argument ?, how the logical structure of the given argument ?; and 3) compare and contrast, (Susan M. Brookhart, 2010). ability to Evaluate To assess the ability of the evaluation, the necessary questions or tasks that represent the ability of justifications or take keptusan. Learners judge something based on a given criteria or criteria formulated by the learners themselves (based on the creativity of learners). There are several types of questions that could be asked to measure the ability to evaluate are: 1) to evaluate the credibility of a source of information, 2) identify the assumptions implicit in the information, and 3) identify methods of rhetoric and persuasive. One example of questions that require evaluation capability is a question of literary criticism. Literary criticism to answer questions like these: how the effectiveness of imagery used by the author? How to create a situation of appeal so as to encourage readers to respond? Similarly, most of the criticism of art or music reviews, restaurant reviews, reviews of the book is the question of evaluation. Similarly, in the natural sciences and social sciences, how strong is the evidence to support the theory (eg, The ability to be creative Questions about mengkreasi demands the ability to solve problems by finding solutions, planning, create a procedure to achieve a particular purpose, or produce something new. Mengkreasi here can be matched to synthesize the original Bloom's Taxonomy, or have a similar meaning with creativity. Norris and Ennis (1989) states that the critical and creative thinking is an important part of a good idea. They distinguish understanding of critical and creative thinking. Creative thinking is logical, productive, and non-evaluative. Critical thinking is logical, reflective and evaluative. But in school assignments in creative and critical thinking are often paired. Problem Solving Ability 38 JISAE. Volume 3 Number 1 February 2017. Copyright © Ikacana Publisher | ISSN: 2442-4919 To solve a problem (problem solving) is generally first performed to identify what the problem is, what might be an obstacle to solving it, and what possible solutions to solve those problems. Bransford and Stein (1984) argued that the measures to solve the problem (problem solving) into five steps are summarized in the acronym IDEAL namely: Identify the problem (identification of problems), define and represent the problem (defining and representing matter), Explore possible strategies (explore strategies that might be), Act on the strategies (solving problems using strategies that may be used), and Look back and evaluate the effects of your activities (an evaluation of the effect as a result of activities undertaken). Indicators complete success in solving problems can be contained in the section. Miscellaneous rubric can be used in the assessment of problem solving ability. The advantage of using the general rubric of problem solving, the students can see the ability to think what is judged according to the learning objectives. With these sections, students are expected to develop the concept of a better problem solving. This means that teachers should be careful to develop rubrics, in order to define the problem solving of teaching lessons. In the form of multiple choice questions, the answer choices represent the ability of learners in solving problems. While the matter of narrative form that requires the ability to construct a response, Susan M. Brookhart (2010) states that there are various types of problems that can be presented to assess problem solving ability, among others: a) the problem of structured and unstructured, meaning that more and more open decision-making means that more and more unstructured problems; b) open-ended problem, a problem that leads to the open answers (many solutions). Reasoning ability Reasoning ability (argued) learners can be honed and developed at school and even at an early age children though. Questions that test the ability to argue generally begins with the question "why". Argued ability can be built on two basic principles of deductive reasoning that the arguments and inductive. (Brookhart, 2010). Deductive argument is the ability to build an argument that starts from the principle to the example of principles (general to specific). For example, the principle of the mathematical sum a + b = b + a, then it must be true for the 6 + 2 = 2 + 6. In deductive arguments, questions starting from one or several premises (the basis for making the argument), then use the argument to draw conclusions. If the premise is wrong, then the conclusion is also invalid. High-level thinking skills require deductive argument that aims to decide whether an element is a member of a category or not. Inductive argument is the opposite of deductive argument: the ability to argue that begins from a data, a specific instance or multiple instances leading to the principle (specifically to the public). For example, in the measurement of classical analysis, identify the theme of literary works and support the theme by evidence is an example of an inductive argument. Another type of inductive argument is to use the analogy of a pattern. This argument builds upon resemblance or similarity of two phenomena or more so that it can serve as the basis for drawing conclusions (conclusion). Based on the above, it can be concluded that in general the ability of argument can be constructed from deductive and inductive arguments. Deductive argument is built on the premises, while the inductive argument is built based on data or specific examples. Both arguments are mutually supportive and very important to test the consistency of an idea 39 that was put together. The accuracy of these arguments election largely depends on the context or the problems presented. Steps compile assessments HOTS In the context of the preparation of the assessment HOTS for daily tests, mid-term assessments, a final assessment exam semester or school-based assessment in general do Competency (KD). Therefore, the steps of the preparation of the assessment are as follows HOTS (Widana, I Wayan, 2016). 1. Analyzing KD which can be made HOTS questions. Not all KD can be made about HOTS, some KD only requires the ability to remember, understand or implement it. Choose KD which requires the ability to analyze, evaluate or creating. 2. Develop grille matter. Grating about the signs were used as a benchmark to write about. In general, grating matter illustrates the KD to be measured, the subject matter that will be tested, the indicators about, and the cognitive level to be measured (eg grating about HOTS can be seen in Table 2). 3. Write down the items on the card problem. Questions written must be in accordance with the criteria established in the lattice matter. Problems HOTS typically in the form of cases, so it must first be considered appropriate stimulus and contextual. The selection of a matter should consider the dimensions of the thought process that will be measured (to analyze, evaluate or create). Sample question cards are shown in Table 3. Writing items should be guided by the rules of writing items. 4. Determining the answer key (the form of multiple choice questions) or arrange rubric / scoring guidelines (shape about the description). Key answer is the correct answer choice or the most correct. While the choice of wrong answers are called detractors. On the matter of narrative form, scoring guidelines should contain key measures (objective description). Each key steps were answered correctly were given a score of 1 or 0 if answered wrong / no answer. On the matter of non- objective description generally contains certain criteria that must be met are called sections. Differences accuracy of the answers in the rubric expressed in the form of a certain gradation, for example, a score of 3 states highly accurate, 2 accurately, one less accurate or inaccurate 0. 5. Perform qualitative analysis. Qualitative analysis aims to look at the suitability of items with item writing conventions. The results of the analysis there are three types accepted, accepted with a repair, or rejected. Those items are accepted when all components of the analysis in accordance with the rules of writing items. Those items are accepted with repairs if a mismatch lies only in the aspect of construction or language. While the items declared rejected or discarded if a mismatch items contained in the material aspect. Only items received and accepted by the improvements that can be tested (quantitative analysis). Format qualitative analysis can be seen in Table 4 and 5. 6. Quantitative analysis. Quantitative analysis aims to determine the characteristic includes distinguishing item, item difficulty level, function or absence of humbug, and guesses (special form of multiple choice questions). Quantitative analysis can be done using software. Only items that meet the requirements can be tested or put into question bank. 40 JISAE. Volume 3 Number 1 February 2017. Copyright © Ikacana Publisher | ISSN: 2442-4919 Benefit Assessment HOTS Susan M. Brookhart (2010) suggests that the benefits of HOTS assessment are as follows. 1. Increase motivation to learn. Often teachers failed to raise the motivation of learners as learning material in class is abstract and theoretical. Therefore, teachers should be able to connect the subject matter in the classroom with real-world context. Assessment HOTS typically in the form of cases, natural phenomena, or contextual issues in daily life that showed linkage of learning materials in the classroom to real-world context. Thus learners can feel that learning in the classroom is very useful to solve problems in everyday life. This is expected to increase the motivation of learners. 2. Improving achievement of learning outcomes. The achievement of learning outcomes is strongly influenced by the motivation of learners. Marhaeni (2005) in his research found that students who have learning motivation tend to have higher academic achievement as well. Therefore, the study suggested teachers able to raise the motivation of learners that learning outcomes can be improved. Furthermore Widana, I Wayan (2016) suggested that one of the benefits HOTS assessment is to improve the competitiveness of students both nationally and internationally. Critical and creative thinking skills gained through work experience HOTS assessment, will have an impact on the creation of habits (habit) positive on problem solving abilities. Troubleshooting can be done according to procedure, using logics, based on the argument (reason) that is logical and reasonable, and accompanied with proof (evidence) to strengthen the findings in problem solving (problem solving). The following are examples of the format of the lattice, about cards and qualitative analysis about HOTS format adapted from the Directorate of High School (2016). Table 2. Format grille Problem HOTS Subjects : .............................................. No. Basic competencies Subject matter Baha / Class indicator Problem Cogniti ve level shape Proble m Table 3. HOTS Problem Card Format Subjects : ........................................ Class / Semester : ........................................ Basic competencies : Subject matter : indicator Problem : Cognitive level : Item Problem: 41 Key to Answer / Manual scoring: Table 4. Assessing instruments Problems HOTS (Multiple Choice) Developer Name Problem : ...................... Subjects : ...................... No. Aspects examined grain Problem 1 2 ... A, 1. Matter Problem accordance with the indicator. □ □ 2. Problem does not contain elements SARAP3K (Tribe, Religion, Race, Anatargolongan, pornography, Politics, Propopaganda, and Violence). □ □ 3. Problem using an interesting stimulus (new, encouraging learners to read). □ □ 4. Problem using contextual stimuli (pictures / graphics, text, visualization, etc., in accordance with the real world) □ □ 5. Problem measure the level of cognitive reasoning (to analyze, evaluate, create). □ □ 6. The answer is implied in the stimulus. □ □ 7. Problem is not routine. □ □ 8. Homogeneous and logical answer choices. □ □ 9. Each question there is only one correct answer. □ □ B. 10. Construction The subject matter defined by a short, clear and unequivocal. □ □ 12. The formulation of the subject matter and response options are statements that needed it. □ □ 13. The subject matter did not give instructions to the answer key. □ □ 14. The subject matter is free from double negative statement. □ □ 15. Pictures, graphs, tables, diagrams, or the like is clear and functioning. □ □ 16. The length of the answer choices are relatively the same. □ □ 17. Choice answers do not use the expression "all the answers to the above one" or "all the answers to the above are true" and the like. □ □ 18. Answer choices that shaped figure / time arranged in order of size numbers or chronology. □ □ 19. Those items are not dependent on another item. □ □ C, 20. Language Using appropriate language to Indonesian rule, for regional languages and foreign languages according the rule. □ □ 42 JISAE. Volume 3 Number 1 February 2017. Copyright © Ikacana Publisher | ISSN: 2442-4919 No. Aspects examined grain Problem 1 2 ... 21. Do not use the language of the applicable local / taboo. □ □ 22. Problem using a communicative sentence. □ □ 23. Answer choices does not repeat the word / group said the same, except one unified understanding. □ □ Table 5. Assessing instruments Problems HOTS (Description) Developer Name Problem : ...................... Subjects : ...................... No. Aspects examined grain Problem 1 2 ... A. 1. Matter Problem accordance with the indicator (requires a written test description form). □ □ 2. Problem does not contain elements SARAP3K (Tribe, Religion, Race, Anatargolongan, pornography, Politics, Propopaganda, and Violence). □ □ 3. Problem using an interesting stimulus (new, encouraging learners to read). □ □ 4. Problem using contextual stimuli (pictures / graphics, text, visualization, etc., in accordance with the real world) □ □ 5. Problem measure the level of cognitive reasoning (to analyze, evaluate, create). □ □ 6. The answer is implied in the stimulus. □ □ 7. Problem is not routine. □ □ B. 8. Construction The formulation of the sentence problems or questions using words or commands that question demands an answer to unravel. □ □ 9. Contains clear instructions on how to do the problem. □ □ 10. There are guidelines for scoring / sections in accordance with the criteria / sentences containing the keywords. □ □ 11. Pictures, graphs, tables, diagrams, or the like is clear and functioning. □ □ 12. Those items do not depend another item. □ □ C. 13. Language Using appropriate language to Indonesian rule, for regional languages and foreign languages according the rule. □ □ 14. Do not use the language of the applicable local / taboo. □ □ 15. Problem using a communicative sentence. □ □ 43 Note: 1) Fill a check mark () in the column or kompunen appropriate aspect. 2) Fill a cross (X) in the column or kompunen aspects that do not fit. CONCLUSION HOTS assessment is an assessment that has the following characteristics: measuring the high-level thinking skills (analyzing, evaluating and creating), based on contextual issues (typically in the form of cases), and are not routine (not familiar). Steps compose about HOTS namely: a) analyze the KD that can be created problems HOTS, b) arrange gratings matter, c) write down the items on the card matter, d) determining the answer key (the form of multiple choice questions) or compose rubric / scoring guidelines (shape about the description), e) conduct a qualitative analysis, and f) perform quantitative analysis. HOTS assessment benefits are: 1) increase the motivation of learners for the assessment HOTS can connect the subject matter in the classroom with real-world contexts so that learning is felt more meaningful; 2) improve learning outcomes for the assessment HOTS can train the way learners think creatively and critically, not just being able to memorize or understand any factual knowledge and concepts; and 3) improving the competitiveness of learners both at national and international level, because through work experience HOTS assessment of learners will have the ability to think critically and creatively better. REFERENCES Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s Taxonomy of Educational Objectives (Complete ed.). New York: Longman. Arter, Judith A.; Jennifer R. Salmon. (1987). Assessing Higher Order Thinking Skills: A Consumer's Guide. Washington, DC: Test Center of the Northwest Regional Educational Laboratory. Australian Council for Educational Research. (2015). Developing Higher Order Thinking Skills. Melbourne: ACER. Bernie Trilling. (2005). Toward Learning Societies And The Global Challenges For Learning–With-ICT. California: Oracle Education Foundation. Biggs, J. B., & Collis, K. F. (1982). Evaluating the quality of learning: The SOLO taxonomy. New York: Academic Press. Bransford, J. D., & Stein, B. S. (1984). The IDEAL problem solver. New York: W. H. Freeman. Brookhart, Susan M. (2010). How to Assess Higher-Order Thinking Skills in Your Classroom. Alexandria, VA: ASCD. 44 JISAE. Volume 3 Number 1 February 2017. Copyright © Ikacana Publisher | ISSN: 2442-4919 Edi Istiyono, Djemari Mardapi, dan Suparno. (2014). “Pengembangan Tes Kemampuan Berpikir Tingkat Tinggi Fisika (PysTHOTS) Peserta Didik SMA”. Jurnal Penelitian dan Evaluasi Pendidikan Tahun 18, Nomor 1. Ennis, Robert H. (1993). Critical Thinking Assessment. Theory Into Practice, Vol. 32, Number 3. College of Education, The Ohio State University. Joke Voogt & Natalie Pareja Roblin. (2010). The 21st Century Skills. Discussion Paper. University of Twente. King, Ludwika Goodson, and Faranak Rohani. (2012). Higher Order Thinking Skills: Definition, Teaching Strategies, Assessment. http://www.cala.fsu.edu/files/higher_order_thinking_skills.pdf Marhaeni, AAIN. (2005). Pengaruh Asesmen Portofolio dan Motivasi Berprestasi dalam Belajar Bahasa Inggris terhadap Kemampuan Menulis dalam Bahasa Inggris. Disertasi. Jakarta: UNJ. Marzano, R. J., & Kendall, J. S. (2007). The new taxonomy of educational objectives (2nd ed.). Thousand Oaks, CA: Sage. Marzano, R. J., Pickering, D., & McTighe, J. (1993). Assessing student outcomes: Performance assessment using the dimensions of learning model. Alexandria, VA: ASCD. Nizam. (2016). “Penilaian Pendidikan”. Paparan tentang Higher Order Thinking Skills (HOTS), (disajikan pada Workshop Penyusunan Soal HOTS Direktorat Pembinaan SMA, di Bogor). Norris, S. P., & Ennis, R. H. (1989). Evaluating critical thinking. Pacific Grove, CA: Critical Thinking Press & Software. PISA. (2016). PISA 2015: Results in Focus. OECD. Robert Swartz and Carol McGuinness. (2014). Developing and Assessing Thinking Skills. Boston: The International Baccalaureate Organisation. Webb, N. L. (2002). Alignment study in language arts, mathematics, science and social studies of state standards and assessments for four states. Washington, DC: Council of Chief State School Officers. Widana, I Wayan. (2016). Penulisan Soal HOTS untuk Ujian Sekolah. Jakarta: Direktorat Pembinaan SMA.