210 | JISAE. Volume 6 Number 2 September 2020. https://doi.org/10.21009/JISAE JISAE (Journal of Indonesian Student Assessment and Evaluation) ISSN : P-ISSN: 2442-4919│E-ISSN: 2597-8934 Vol 6 No 2 (2020) Website : http://journal.unj.ac.id/unj/index.php/jisae DEVELOPMENT OF ASSESSMENT INSTRUMENTS BASED ON HOTS AT SURABAYA VOCATIONAL HIGH SCHOOL Rudi Hartono 1 , Triesninda Pahlevi 2 12State University of Surabaya ABSTRACT This study aims to produce assessment instruments according to Higher Order Thinking Skills (HOTS) in General Administration subjects at Basic Competencies 3.9 The Implementation of Office Layout, and 3.10 The Implementation of Communication in the Workplace as many as 40 items of multiple-choice, and to decide the feasibility level of the questions in measuring students' HOTS based on the validation results from the material expert, evaluation expert, and linguist. This study is Research and Development (R&D) adapted Sugiyono's development model that has been adopted into 5 stages: potential and problems, data collection, product design, design validation, and design revision. Then, the research instruments used by the researcher were the validation sheet from the expert of material, evaluation, and the linguist. Furthermore, the validation result from the material expert was 98.3% (very strong). Besides, the validation result of the evaluation expert was 100% (very strong). Moreover, the validation results from the linguist were 98.7% (very strong). Therefore, the average validation result was 99% with the interpretations of criteria were "very strong" and feasible to use. In conclusion, the development of assessment instruments based on HOTS at Surabaya vocational high school was feasible to use in measuring students' HOTS. Keywords: An Assessment Instrument Development, General Administration, HOTS. Address for Correspondence: rudihartono16080314032@mhs.unesa.ac. id1, triesnindapahlevi@unesa.ac.id2 INTRODUCTION Education is a human effort in developing or enhancing the talents and skills that have been born at birth to be more useful. Education must always be developed in line with the development of the existing era, therefore education at this time is demanded to create quality human beings and be able to adapt to the era of the industrial revolution 4.0. According to Lestari (2018), the quality of human resources can be determined by improving the quality of education. The Government of Indonesia seeks to improve the quality of education by implementing the revised 2013 Curriculum which is applied at elementary level up to senior high school. The revised 2013 curriculum or also can be called the K13 revision has a theme that is a curriculum that can produce a nation that has good and effective creativity, productivity, and innovation by strengthening attitudes, developing skills, and integrated knowledge (Mulyasa, 2014). Based on this theme, the characteristics of the 2013 revised curriculum are making students more active and able to develop critical thinking, systematic, creative, objective, and chronological in the learning process. The statement agrees with Apandi (2017) which states that the 2013 curriculum has the characteristics of requiring students to be active and can improve their ability to think at a high level during learning activities. In the implementation of the revised 2013 curriculum, HOTS-based questions are demanded to be developed in schools so that students can achieve the objectives of the 2013 http://journal.unj.ac.id/unj/index.php/jisae mailto:rudihartono16080314032@mhs.unesa.ac.id mailto:rudihartono16080314032@mhs.unesa.ac.id mailto:triesnindapahlevi@unesa.ac.id 211 | JISAE. Volume 6 Number 2 September 2020. curriculum. There is one vocational high school in Surabaya thay One of the schools that applies HOTS-based assessments. Based on the results of the preliminary study, this school, which has an A accreditation and ISO 9001-2000 certified, has implemented a revised 2013 curriculum and applied the HOTS based assessment instrument to all students. Ms. Mustipah as a general administration teacher in the Otomatisasi Tata Kelola Perkantoran (OTKP) department explained that the HOTS characteristic assessment instruments had been applied in the of general administration subjects but the numbers were still limited so students were not accustomed to working on HOTS-based questions. As a teacher, the teacher must prepare students to be able to compete in the industrial revolution 4.0 by getting students used to the ability to think at a high level and also be able to make good assessments following opinions Oli (2018) the teacher is the best person and has the right position to assess the student so the teacher must be considered as the student's primary assessor. One way to prepare students' high-level thinking skills is by applying an assessment instrument characterized by HOTS. This is in agreement with Budiman & Jailani (2014), the application of problems characterized by HOTS is an effort to enable students to think critically and skillfully. General administration is a tenth graders subject majoring in Otomatisasi Tata Kelola Perkantoran (OTKP) as well as a basic lesson before students learn subjects about further administration. General administration is also said to be a contextual subject or can be associated with daily activities or events as one of the characteristics of HOTS, so students are expected to know when they are in the workforce. This is in agreement with Sari (2014), Administrative lessons are important because they can provide knowledge and lessons before students enter the workforce. General administration subjects have 12 basic competencies, including Basic Competencies 3.9 Applying Work Layout or Office and 3.10 Applying Communication at Work. These basic competencies are often carried out in the world of work activities, so this basic competency is said to be important in preparing students' ability to enter the workforce. Based on the background above and from the results of the study (Istiyono, Mardapi, & Suparno, 2014; Malik, Rosidin, & Ertikanto, 2018) which states if needed questions characteristic of HOTS to improve students' higher-order thinking skills. Plus the importance of general administration lessons in preparing students before entering the workforce (Sari, 2014). Therefore, there is a need to develop an assessment instrument characterized by HOTS for the General Administration subject of the OTKP Department at Surabaya vocational high school. The instrument is arranged in the form of multiple-choice and has a composition of answers as many as 4 choices with 1 correct answer on each item. The preparation of this assessment instrument is guided by the book (Widana, 2017). This study aims to develop questions or assessment instruments that are characterized by HOTS in general administration subjects in Basic Competencies 3.9 and 3.10 and to determine the appropriateness of assessment instruments developed. Assessment or can also be called assessment is an action/process that is systematic and ongoing to collect information/data about student learning outcomes as a basis for decision making by taking into account predetermined criteria and considerations (Arifin, 2009). Explained in Government Regulation No. 30 Tahun 2013 tentang Standar Nasional Pendidikan Pasal 1 ayat 24 assessment is a process of gathering and processing information to measure the achievement of the learning process of students. Meanwhile according to Uno & Koni (2012) in general assessment or assessment is a process carried out to obtain any form of information or data that can be used as a basis for determining good decisions on the curriculum, students, programs, and policies of educational institutions. Meanwhile according to Pahlevi, Rosy & Ranu (2018) "Assessment is one of the key indicators to determine the success or failure of educators as learning agents and learners before selecting appropriate 212 | JISAE. Volume 6 Number 2 September 2020. targeting methods that are considered appropriate to existing learning conditions so that for the next step of effectiveness, efficiency and power the pull of learning can be well organized and can produce competent learning outcomes that can make the assessment of learning is positive, according to national education goals " which means assessment is an indicator to determine the success of teachers and students before choosing the right targeting method so that the next step can be more effective and efficient and can produce good learning outcomes and create positive learning assessments according to national education goals. It is concluded from the expert opinions above, The purpose of the assessment process in general is to provide an overview of the ability to master the competencies that have been made by students in the learning process and serve as benchmarks to improve the learning process (Uno & Koni, 2012) The statement agrees with Helmawati (2019) which states that the purpose of assessment is to obtain information as material for evaluating the learning process, the level of student learning ability and for the process of improving learning outcomes. Based on the expert opinion above, it can be concluded that the assessment process aims to find out the learning outcomes of a student as well as an evaluation material to improve student abilities and become a benchmark in improving the learning process and activities. According to Arum & Lestari (2019), good assessment occurs because there are quality instruments. After all, the preparation of assessment instruments must pay attention to existing guidelines or principles. According to Helmawati (2019), there are 9 assessment principles, namely valid, fair, open, integrated, objective, continuous and comprehensive, structured, accountable, and based on achievement measures. HOTS is a thought process that requires one to think critically and creatively to solve problems. As the opinion ofHelmawati (2019) HOTS indicators include being able to think critically and creatively. According to (Malik, Rosidin, & Ertikanto, 2018; Oktanisa, 2018; Purbaningrum, 2017) Higher level thinking is a thinking technique that enables people to manipulate information or data received by changing meaning, combining facts and planning them to be synthesized, generalizing and interpreting to conclude. Based on some expert opinions above, HOTS or higher order thinking skill is a mindset that requires a person to think critically and creatively by processing various information and then combining it with known facts so that conclusions can be drawn in solving a problem. According to Krathwohl (in Purbaningrum, 2017) states that there are 3 indicators in the measurement of high-level thinking power, namely: 1) Analyzing. 2) Evaluating, 3) Creating. From these indicators, it can be interpreted that high-level thinking skills are grouped in cognitive aspects of the C4 level (Analyze), C5 (Assess), and C6 (Create) in Bloom's taxonomy developed by Benjamin Samuel Bloom ( Gunawan & Palupi, 2012). An assessment instrument is a tool or technique in the form of tests or notes that can be used to collect and measure data. According to Taufiqurrahman, Heryandi, & Junaidi (2018), An instrument is something that can be used to measure the data collection process. Meanwhile, according to Jannah & Pahlevi (2020), assessment instruments can also be called assessment techniques divided into two forms, namely the form of tests and non-test forms. Based on the expert opinion above, it was concluded that the assessment instrument characterized by HOTS is a technique or tool that can measure a person's high-level thinking skills using either a test or non-test form. The higher order thinking assessment instrument has 3 characteristics (Widana, 2017), namely: 1) the questions can assess students' high-level thinking skills. 2) questions are contextual or under daily life. 3) Questions are made in various forms. Based on the opinions expressed by Widana (2017), the preparation of HOTS- based assessment instruments is carried out in stages starting from 1) Conducting an analysis of Basic Competencies which will be used to create HOTS questions. 2) Make a HOTS-based 213 | JISAE. Volume 6 Number 2 September 2020. question grid. 3) Presenting stimulus questions that are contextual and interesting. 4) Arrange questions by referring to the lattice that has been made. 5) Prepare a list of answer keys and scoring instructions (rubrics). Previous research states that a good assessment process occurs because of the existence of quality assessment instruments (Arum & Lestari, 2019). The application of the HOTS approach can improve learning outcomes in all materials because its application can adjust to the conditions and character of students (Sofyan, 2019), and research results Taufiqurrahman, Heryandi, & Junaidi (2018) mentioned that HOTS assessment instrument is very good to give to students, but the manufacturing process for Religion subjects was very difficult and requires a lot of time. According to Budiman & Jailani (2014), Jannah & Pahlevi (2020) and Istiyono, Mardapi, & Suparno (2014) HOTS-based assessment instrument development in the form of multiple-choice which has been validated by experts or material validators, construction/evaluation and language can be used to measure one's high-level thinking skills. RESEARCH METHODS This research is classified into the type of development research or commonly called R&D by applying the development model according to Sugiyono (Sugiyono, 2017) with 10 stages of research, but in this study is limited to only 5 stages namely starting from 1) potential and problems, 2) data collection, 3) product design 4) design validation, 5) and design revision. This limitation is carried out due to limited time, and place conditions. This research uses quantitative and qualitative data types. Qualitative data are generated from interviews and validation results of assessment instruments in the form of input and suggestions regarding research products made, and quantitative data that is data obtained from processing numbers from validation instruments to determine the feasibility of this product. This study uses interviews and expert validation results sheets as instruments for data collection. The type of interview used is the type of unstructured interview conducted with general administration subject teacher at one of Surabaya Vocational High School. The expert validation sheet is produced from the expert assessment of the material, evaluation, and language by giving a checkmark (√) or a cross (X) then giving criticism and suggestions. Assessment of the three experts or validators based on indicators Widana (2017) following: Material expert assessment indicators, namely 1) Each item uses an interesting stimulus. 2) Questions using contextual stimuli include pictures, graphics, text, or visualizations. 3) Questions are measuring the C4-C6 level (analyzing, evaluating, and creating) according to the level of HOTS-based questions. 4) There is an answer implicit in the stimulus problem. While the evaluation indicators of expert evaluation are 1) There are clear work instructions. 2) The existence of clear scoring guidelines. 3) Each item made does not depend on the answers to other items. While the assessment indicators used by linguists are 1) Using the rules for the preparation of Indonesian Language that is good and right. 2) Using language that is easy to understand and does not cause multiple interpretations. 3) Use communicative sentences. The validation results are processed using descriptive quantitative data analysis techniques using Guttman scale calculation with details that get the category "Yes" will get a value of 1, while those who get the category "No" get a value of 0 (Riduwan, 2015). The results of data analysis from the three experts were then calculated using the method (Riduwan, 2015) : Percentage = 𝑎𝑐𝑞𝑢𝑖𝑠𝑖𝑡𝑖𝑜𝑛 𝑠𝑐𝑜𝑟𝑒 𝑚𝑎𝑥𝑖𝑚𝑢𝑚 𝑠𝑐𝑜𝑟𝑒 𝑥 100% 214 | JISAE. Volume 6 Number 2 September 2020. The results of the validation that have been processed in the form of a percentage are analyzed their level of eligibility by referring to the table of expert validation interpretation criteria according to Riduwan (2015) with the following details: assessment results that get a value of 0% - 20% get the criteria "Very Weak", the value of 21% - 40% get the criteria "Weak", a value of 41% - 60% get the criteria "Enough", a value of 61% - 80% get the criteria "Strong", and the value 81% - 100% get the criteria "Very Strong". RESULTS AND DISCUSSION HOTS characteristic assessment instruments in general administration subjects especially on Basic Competencies 3.9 Applying Office Layout, and 3.10 Implementing Communication at Work uses a development model according to Sugiyono which is limited to only 5 stages, namely the Potential and problem stages, Data collection, Product design, Design validation, and Revision the design. At potentials and problems stages, the researcher chooses the place of research while simultaneously analyzing the potential and problems at this school. This school is a public vocational school in Surabaya and has been accredited with A and ISO 9001-2000 certified with 7 competency skills, also applies the revised K13 and has applied HOTS-based assessment instruments. However, in general administration subjects, especially Basic Competencies 3.9 Applying Work or Office Layout, and 3.10 Applying Communication at Work is still limited. Data collection stage, namely the stage of researchers conducting observations or preliminary studies by conducting unstructured interviews with the general administration teacher at one of the vocational high school in Surabaya. The interview aims to find out about the curriculum and types of assessment instruments applied in this school. The results of the interview stated that this school had implemented the revised K13 curriculum and used the assessment instrument characterized by HOTS but in general administration lessons Basic Competencies 3.9 Applying Office Layout, and 3.10 Applying Communication at Work was still limited, so students are less accustomed to thinking high level to do the questions on the Basic Competence. The product design stage, which is the preparation stage of the HOTS characteristic assessment instrument, consists of 40 items in the form of multiple-choice that has a composition of answers as many as 4 choices of answers with 1 correct answer. Advantages of multiple-choice test questions according to Istiyono, Mardapi, & Suparno (2014), namely: (1) the material covers most of the learning material; (2) the reasoning is objective because the answer must be right or wrong; (3) the correction process is easier and faster. Agree with that, (Arifin, 2009) stated that measuring student learning outcomes can be more complex and objective when using multiple-choice questions. The assessment instruments developed include the cognitive domains C4, C5, and C6. The preparation of this assessment instrument is based on the stages of preparing the HOTS-based assessment instrument according to Widana (2017), the stages are making a problem lattice, using an interesting stimulus, compiling questions that are adjusted to the lattice that has been made, and compiling the answer key. The lattice made consists of basic competencies, indicators of competency achievement, materials, question indicators, cognitive levels, and question numbers. The composition of HOTS questions in Basic Competencies 3.9 Applying Office or Work Layout and 3.10 Implementing Communication in the Workplace which was developed consisting of 23 questions with C4 level, 4 questions about C5 level, and 13 questions about level C6. Design validation phase, at this stage the questions that have been made will be assessed by three validators, namely material validator, evaluation, and language. agree with Jannah & Pahlevi (2020), to determine the feasibility level of the assessment instrument that can be done employing a qualitative test that is obtained from the validator of evaluation, 215 | JISAE. Volume 6 Number 2 September 2020. material, and language. The results of the assessment of the three validators are known as follows: from the material validator a score of 98.7% has a "very strong" criterion, the results of the evaluation validator have a score of 100% that has a "very strong" criterion, and from a language validator a score of 98.3 which has the criteria "very strong". At this stage, the validator also provides criticisms and suggestions for problems that are still not feasible enough to be fixed so that they are suitable for use. The design revision stage is the stage of correcting HOTS-based assessment instruments that still have errors during the design validation stage. The assessment instrument was improved based on criticism and input obtained from the validator before the assessment instrument was said to be feasible and could be used in the field. The feasibility of the instrument characterized by HOTS was determined based on the results of the assessment of the material validator, evaluation, and language. in harmony with opinion Hartini & Sukardjo (2015) which argues that in determining the results of validation must be based on material aspects, aspects of evaluation or construction, and linguistic aspects. Based on the validation results that have been calculated using the Guttman scale and presented in the table below: Table 5 Material Validation Results No. Indicator Percentage (%) 1 Stimulus Feasibility 100 2 The stimulus is contextual 97.5 3 Measuring levels C4, C5, and C6 97.5 4 There is a thesis answer 100 Average 98.7 Source: Researcher Processed Data (2020) Table 6 Evaluation Validation Results No. Indicator Percentage (%) 1 There are work instructions 100 2 There are Scoring Guidelines 100 3 Each Item Does Not Depend on the Answer of Other Problem Items 100 Average 100 Source: Researcher Processed Data (2020) 216 | JISAE. Volume 6 Number 2 September 2020. Table 7 Language Validation Results No. Indicator Percentage (%) 1 Using Good Indonesian Language Compilation Rules 100 2 Language is easy to split 97.5 3 Using Communicative Sentences 97.5 Average 98.3 Source: Researcher Processed Data 2020) Validation results that have been presented in the form of a percentage, then analyzed by referring to the validation expert interpretation criteria and the following results are obtained: Source: Researcher Processed Data (2020) Figure 1 Results of Validation Interpretation Criteria Based on the data in the figure above, it is known that the material expert gives an assessment of 98.7% with the criteria "very strong", evaluation of the expert evaluation by 100% with the criteria "very strong" and the linguist gives a value of 98.3% with the criteria "very strong". Based on the assessment that can be obtained from the three validators, an average value of 99% is obtained with the criteria of "very strong". The validator also gave criticism and suggestions, and also stated that the HOTS-based assessment instruments in general administration subjects at Basic Competencies 3.9 Implementing Office Layout and 3.10 Applying Communication at Work is appropriate for use as a daily test item for students majoring in Otomatisasi Tata Kelola Perantoran (OTKP). CONCLUSION Based on the results of the discussion, it can be obtained into 2 conclusions from the research development of assessment instruments based on Higher Order Thinking Skills (HOTS) general administration subjects majoring in office management automation at one of the vocational high school in Surabaya, namely: 1) assessment instruments for basic competencies 3.9 Implementing Office Layout and 3.10 Implementing Communication at Work has been developed with the characteristics of HOTS A total of 40 items in the form of multiple-choice test questions that have a composition of answers as many as 4 choices of answers with 1 correct answer. 2) the assessment instruments developed have been declared feasible by the expert validator of the material, evaluation, 98,7% 100% 98,3% Result of Validation Interpretat ion 217 | JISAE. Volume 6 Number 2 September 2020. This study has a limitation that is the assessment instrument was only developed in the form of multiple-choice of 40 items. Assessment instruments are made only in the realm of cognitive analysis (C4), evaluating (C5), and creating (C6). The development model used is the development of Sugiyono which consists of 10 stages but this assessment instrument uses only 5 stages namely potential and problems, data collection, product design, design validation, and design revision. Based on the discussion above, the researcher provides suggestions for further research to develop questions in several forms and greater numbers. Development is carried out up to a broad trial phase, and the assessment process can be carried out using applications so that students are happier in the assessment process. REFERENCES Apandi, I. (2017). Tiga Agenda Penting Implementasi Kurikulum 2013. Arifin, Z. (2009). Evaluasi Pembelajaran. Bandung: Remaja Rosdakarya. Arum, S., & Lestari, P. (2019). Pengembangan Instrumen Asesmen Higher Order Thinking SKILL ( HOTS ) pada Materi Himpunan Kelas VII SMP. 2682, 111–120. Budiman, A., & Jailani. (2014). Pengembangan Instrumen Asesmen Higher Order Thinking Skill (HOTS) Pada Mata Pelajaran Matematika SMP Kelas VIII Semester 1. Junal Riset Pendidikan Matematika, 1(November 2014), 139–151. Gunawan, I., & Palupi, A. R. (2012). Taksonomi Bloom - Revisi Taksonomi Bloom Ranah Kognitif: Kerangka Landasan Untuk Pembelajaran, Pengajaran, Dan Penilaian. 2(1), 98–117. Hartini, & Sukardjo. (2015). Pengembangan Higher Order Thinking Multiple Choice Test Untuk Mengukur Keterampilan Berpikir Kritis IPA Kelas VII SMP/MTs. 1(April), 86– 101. Helmawati. (2019). Pembelajaran dan Penilaian Berbasis HOTS. Bandung: Remaja Rosdakarya. Istiyono, E., Mardapi, D., & Suparno. (2014). Pengembangan Tes Kemampuan Berpikir Tingkat Tinggi Fisika (PysTHOTS) Peserta Didik SMA. Jurnal Penelitian Dan Evaluasi Pendidikan, 1–12. Jannah, K., & Pahlevi, T. (2020). Pengembangan Instrumen Penilaian Berbasis Higher Order Thinking Skills Berbantuan Aplikasi “ Kahoot !” Pada Kompetensi Dasar Menerapkan Penanganan Surat Masuk dan Surat Keluar Jurusan OTKP di SMK Negeri 2 Buduran. 8(2018), 108–121. Lestari, N. D. (2018). Penerapan Kurikulum 2013, Meningkatkan Kualitas Pembelajaran Ekonomi di SMA Negeri Se_kota Palembang. Jurnal Manajemen Pendidikan, 2(1), 68– 79. Malik, A., Rosidin, U., & Ertikanto, C. (2018). Pengembangan Instrumen Asesmen HOTS Fisika SMA Menggunakan Model Inkuiri Terbimbing. 3(1), 11–25. Mulyasa. (2014). Pengembangan dan Implementasi Kurikulum 2013. Bandung: PT Remaja Rosdakarya. Oktanisa, L. (2018). Pengembangan Asesmen Soal Berbasis Higher Order Thinking Skills Pada Mata Pelajaran Ekonomi. 6(3), 355–361. Oli, M. C. (2018). THE ASSESSMENT PRACTICES BY CONTENT FACULTY , STUDENT-TEACHING SUPERVISOR AND COOPERATING MENTORS OF PRE- PROFESSIONAL MATHEMATICS TEACHERS IN STATE UNIVERSITIES IN NORTHEASTERN PHILIPPINES. JISAE, 4(2), 60–82. Pahlevi, T., Rosy, B., & Ranu, M. E. (2018). A Scientific Approach Based on Portfolio Assessment for Autonom Problem Solving. International Journal of Educational Research Review, 29–36. Retrieved from www.ijere.com 218 | JISAE. Volume 6 Number 2 September 2020. Purbaningrum, K. A. (2017). Kemampuan Berpikir Tingkat Tinggi Siswa SMP Dalam Pemecahan Masalah Matematika Ditinjau Dari Gaya Belajar. Jurnal Penelitian Dan Pembelajaran Matematika, 10(2), 40–49. https://doi.org/10.30870/jppm.v10i2.2029 Riduwan. (2015). Skala Pengukuran Variabel-variabel Penelitian. Bandung: Alfabeta. Sari, rika P. (2014). Pengembangan Buku Ajar Berbasis Model Pembelajaran Discovery Learning Pada Mata Pelajaran Administrasi Umum Kelas X OTKP 1 di SMK Negeri Mojoagung. 45. Sofyan, F. A. (2019). Implementasi Hots Pada Kurikulum 2013. Inventa, 3(1), 1–9. https://doi.org/10.36456/inventa.3.1.a1803 Sugiyono. (2017). Metode Penelitian Kuantitatif, Kualitatif dan R&D. Bandung: Alfabeta. Taufiqurrahman, Heryandi, M. T., & Junaidi. (2018). Pengembangan instrumen Penilaian Higher Order Thinking Skill pada Mata Pelajaran Pendidikan Agama Islam. 2(April), 199–206. Uno, H. B., & Koni, S. (2012). Assessment Pembelajaran. Jakarta: Bumi Aksara. Widana, I. W. (2017a). HIGHER ORDER THINKING SKILLS ASSESSMENT ( HOTS ). JISAE, 3(1), 32–44. Widana, I. W. (2017). Penyusunan Soal Higher Order Thinking Skill (HOTS). 1–40.