8 |JISAE (Journal of Indonesian Student Assessment and Evaluation) |Volume 9 Number 1 https://doi.org/10.21009/JISAE JISAE (Journal of Indonesian Student Assessment and Evaluation) ISSN : P-ISSN: 2442-4919│E-ISSN: 2597-8934 Vol 9 No 1 (2023) Website : http://journal.unj.ac.id/unj/index.php/jisae ORAL ASSESSMENT IN TRIGGERING STUDENT HIGHER-ORDER THINKING SKILLS (HOTS) I Wayan Eka Mahendra1, Institut Pariwisata dan Bisnis Internasional ABSTRACT Students' higher-order thinking skills (HOTS) due to the application of oral Assessment and Assessment are usually compared using experimental research in two separate groups. The experimental group consisted of 31 students, and the control group of 32 students was taken using a simple random sampling technique from a population of 315 students. Students' HOTS data were collected utilizing a test consisting of 5 items tested for validity, and the reliability coefficient was calculated. The collected data were then analyzed using a test (t-test), and the classical assumption test had previously been carried out. It turns out that the HOTS of students who are given an oral assessment is better than the HOTS of students who are given a regular assessment. Even though students are more afraid to face oral Assessment, they consider oral Assessment more beneficial than ordinary Assessment. Keywords: Higher Order Thinking Skill; Oral Assessment Address for Correspondence: 1. ekamahendra@ipb-intl.ac.id INTRODUCTION The learning that occurs is an intervention that the teacher deliberately carries out to improve the situation in the classroom. Ames (1992) warns that a comprehensive approach to classroom intervention is essential because changes in the classroom are a precursor to changes in the school environment. The approach in question is not only a method, model, or learning strategy but also an assessment approach used by teachers. The most effective intervention to involve students in the learning process is instructions that encourage action beyond what is carefully written in the teacher's lesson plan (Ferreri & O'Connor, 2013, Hammonds & Schwarze, 2019). From an assessment point of view, a variety of assessments are needed that have been carefully designed by the teacher, such as tests, questionnaires, observation guidelines, and assessments that arise spontaneously during the learning process. This Assessment is known as an oral assessment. Traditionally until now, oral Assessment utilizes the Assessment of the listener, in this case, the teacher and students, to evaluate students' abilities (Park, 2020). Oral Assessment has advantages over paper and pencil test-based assessments that dominate the current assessment system from elementary to tertiary levels. This Assessment can help students integrate their cognitive abilities and provide feedback from experience, requiring immediate integration and reflection (Hammonds & Schwarze, 2019). Oral Assessment outside the law, medicine, or architecture is considered an alternative assessment (Joughin, 1998). According to the Oxford English Dictionary, an oral Assessment is an assessment in which the http://journal.unj.ac.id/unj/index.php/jisae 9 student's response to the assessment task is verbal, in the sense that it is expressed or conveyed by speech rather than in written form. Thus the Assessment is considered an oral assessment as long as the student's response is verbal. Furthermore, Joughin (1998) revealed four components that can be assessed through oral Assessment in the main content dimensions: knowledge and understanding, applied problem solving abilities, interpersonal skills, and interpersonal qualities. Swank (2012) reveals the importance of using various creative and innovative strategies in classroom learning in transforming passive learning into active learning, and oral Assessment is well suited for this purpose due to the flexible nature of design and application. There are still many teachers whose understanding is still lacking on how to assess student learning outcomes (Minton, Gibson & Morris, 2016), moreover sharing the standard methods used by teachers in assessing student abilities is still not valid (Sadler, 2009). It is undeniable that validity and reliability have always been an important issue in any form of Assessment. Especially for oral assessments, concerns about validity and reliability have been at the forefront of discussions of this assessment format since at least 1929 (Joughin, 1998). The validity of this oral Assessment can be relied on especially in accessing applied problem solving skills, interpersonal competencies, or personal qualities (Joughin, 1998). Various studies have been conducted on the effectiveness of oral Assessment in improving students' abilities. Tekian and Yudowsky (2009); Banning (2008); Durning et al. (2013); Richardson et al. (2017); Turner & Davila-Ross (2015) found that oral Assessment was able to explore the skills of nursing participants such as critical reasoning, problem solving, ethics, demonstrating the ability to express ideas, synthesize material, and think independently. Banning (2008); Levett-Jones et al. (2011) found that oral Assessment is able to measure different matrices of abilities such as cognitive, attitudes, values, skills, critical thinking, insight, and factual knowledge in various disciplines such as: language, law, and counselor education. In general, research on oral Assessment is still rarely done (Huxham, Campbell, & Westwood, 2012). This is evident from the study conducted by Hounsell et al. (2007) who reviewed the literature on innovative Assessment. Of the 317 papers reviewed, only 31 discussed oral Assessment, and in this category only 13% or approximately 4 literatures discuss the use of oral exams; oral group presentation METHOD This research is classified as quasi-experimental research with a non- equivalent control group design. Involving two groups, namely the experimental group and the control group but the control group does not strictly control other variables that affect the dependent variable (high order thinking skills). By involving a sample of 64 students taken by simple random sampling technique from 315 students of class VIII SMP Negeri 1 Bangli. Sampling was carried out randomly because of the information from the principal that class VIII students were homogeneously distributed without any superior or non-superior class. The distribution of the population in this study is presented in the following tablel 1: 10 |JISAE (Journal of Indonesian Student Assessment and Evaluation) |Volume 9 Number 1 Table 1: Distribution of VIII Grade Students of SMP N 1 Bangli Who Becomes the Research Population No Class Amount Total Male Female 1 VIII A 16 15 31 2 VIII B 13 18 31 3 VIII C 16 16 32 4 VIII D 14 18 32 5 VIII E 15 17 32 6 VIII F 15 17 32 7 VIIIG 14 18 32 8 VIII H 16 16 32 9 VIII I 15 16 31 10 VIII J 14 16 30 Total 148 167 315 Of the ten existing classes, namely from class VIII A to class VIII J, 2 classes were selected as research samples. By making a roll of paper, first 2 papers were selected as research samples, then from the 2 classes were further divided into two, class H was selected as the experimental group with 32 students and class B was selected as the control group with 31 students. The experimental group was given treatment in the form of an oral assessment in the learning setting applied by the teacher, while the control group was not given treatment but was given an assessment that is usually done by the teacher during learning. Thus, the treatment in this study is a form of Assessment, not a learning model. The data collected is data on high order thinking skills (HOTS) of students in the experimental group and the control group. The data was collected using the HOTS test (analysing, evaluating, and creating) which consisted of 5 questions covering mathematical material in the form of circles, shapes, straight lines, and triangles that had been tested for validity and calculated the reliability coefficient. The collected data was tested with parametric statistics in the form of a mean difference test (t-test), which had previously been carried out with the classical assumption test in the form of a normality test for data distribution and a homogeneity of variance test. RESULTS AND DISCUSSION Result All data analysis was carried out using SPSS 26.0 for windows. The mean difference test (t-test) is a statistic used as a data analysis tool. Before testing the hypothesis, which is preceded by a classical assumption test, the data concentration (tendency central) and data dispersion (dispersion) of the research results are presented, or what is known as descriptive statistics. It aims to describe or provide an overview of the object under study through sample data (Sugiyono, 2019). There were two groups of data that were analyzed, namely the HOTS data group for students who were given an oral assessment, hereinafter referred to as Y1, and the HOTS data for students who were given a regular assessment, hereinafter referred 11 to as Y2. The following table presents the results of descriptive statistical analysis of the two groups of data. Table 2: Description of HOTS Data for Experimental Group and Control Group Statistic Experimental Group Control Group Jumlah 2490.00 1995.00 Rata-Rata 80.32 62.34 Median 82.00 64.50 Modus 80.00 68.00 Standar Deviation 7.89 8.44 Varians 62.35 71.33 Jangkauan 32.00 38.00 Nilai Minimum 63.00 42.00 Nilai Maksimum 95.00 80.00 From Table 2, it can be seen that there are differences in data descriptions between Y1 and Y2 starting from the number, average, median, mode, standard deviation, variance, range, minimum value, and maximum value. Mathematically, there is a very high difference between the average HOTS value of the experimental group and the average HOTS value of the control group. The average HOTS score of the experimental group students is 80.32 while the average HOTS score of the control group students is 62.34. There is a difference of approximately 17.98 points. In other words, the experimental group has a higher average HOTS value than the control group. To make sure that the difference is significant, it is necessary to do statistical tests, but before that, a prerequisite analysis test (classical assumption test) was carried out in the form of a normality test for data distribution and a homogeneity test of variance. The normality test was conducted to ensure that the statistical test used in the test using the t-test which is a parametric statistic can be performed. The normality test in this study used the Kolmogorov-Smirnov module on both groups of students' HOTS score data. Kolmogorov-Smirnov test analysis shows that the sig. > 0.05 for both groups of data, namely the HOTS data of the experimental group students (Y1) of 0.200 and the control group (Y2) of 0.134. This means that H0 is accepted (failed to be rejected), both groups of data have a normal distribution. While the homogeneity of variance test is intended to ensure that the differences obtained from the t-test really come from differences between groups, not due to differences within groups. From the results of the analysis obtained the value of sig. > 0.05 or 0.741 > 0.05 in the Based on Median and with adjusted column; This means that both groups come from populations that have the same or homogeneous variance. Based on the results of the classical assumption test, it can be concluded that the student's HOTS data comes from a population that is normally distributed and has the same or homogeneous variance. Thus, hypothesis testing with t-test can be done. From the output of the t-test, the t-test significance value for the assumption of equal variances (equal variances assumed) and the two-tailed test (two tailed) is 0.001. So the value of sig. < of 0.05 or 0.001 < 0.05, this means that H0 is rejected and H1 is accepted. It is said that there is a difference in HOTS between students who are given an oral assessment and students who are given a regular assessment (paper and pencil test). Thus, it can be concluded that there is an effect of applying 12 |JISAE (Journal of Indonesian Student Assessment and Evaluation) |Volume 9 Number 1 oral Assessment on high order thinking skills (HOTS) of class VIII SMP Negeri 1 Bangli. Discussion The results of data analysis using t-test showed that there were differences in high order thinking skills between students who were given an oral assessment and students who were given a conventional assessment. This indicates that in order to obtain maximum high order thinking skills of students, it is necessary to improve the quality of interventions carried out by teachers, in this case the assessment approach used. The superiority of oral Assessment compared to conventional Assessment can be seen from the average high order thinking skills of students. Where the average high order thinking skill of the experimental group is 80.32 which is higher than the average high order thinking skill of the control group of 62.34. The advantages of oral Assessment compared to conventional assessments in improving students' HOTS are not only limited to a theoretical framework but have been empirically tested in the field through this research. HOTS can be interpreted as thinking skills that are non-algorithmic, tend to be complex, produce many solutions, solve problems with full effort (Resnick, 1992), critical thinking skills, creative thinking (Krulik & Rudnick, 1999), analyze, synthesize, and create (Anderson & Krathwohl, 2001). It turns out that the characteristics of high order thinking skills as described can be trained by applying oral Assessment during the learning process. The results of this study are in line with the results of research conducted by Joughin (1998) which shows that one of the abilities that can be improved through the application of oral Assessment is applied problem solving skills. The category of applied problem solving is described as the ability to solve problems on their own, think professionally, think quickly and diagnose problems in new situations, problem solving skills, and creative thinking skills. Furthermore Kaplowitz et al. (1996) claim that the main advantage of oral Assessment is that teachers are able to ask students a series of related questions that can test not only basic level knowledge, but more complex knowledge, as well as how well they can apply the knowledge they have. This shows how effective oral Assessment is in training and improving students' HOTS. Huxhama, Campbellb, and Westwoodc (2012) conducted a study by comparing oral Assessment and written Assessment. This research examines performance and attitudes towards oral and written assessments using quantitative and qualitative methods. The results showed a very significant difference in the average posttest scores given, where the performance of students who were given an oral assessment was better than the performance of students who were given a written assessment. There is no evidence that one of the groups is harmed by the application of the oral Assessment. Meanwhile, students' attitudes about these two types of Assessment indicate that oral Assessment is considered more useful than written Assessment, even though they tend to be more nervous and anxious in dealing with oral assessments. With the oral Assessment students tend to prepare themselves better than the written Assessment. Oral Assessment is considered capable of increasing students' professionalism in learning and understanding the material provided by the teacher. Students feel embarrassed if they are not able to answer the teacher's questions orally. This more mature preparation tends to motivate students to study harder, especially in improving their HOTS. Oral Assessment is a powerful and exclusive 13 tool in helping students build a professional identity (Huxhama, Campbellb, and Westwoodc, 2012). CONCLUSION The conclusion from the findings of the research conducted is that there is a significant effect of the application of oral Assessment on high order thinking skills (HOTS) of class VIII students of SMP Negeri 1 Bangli. In increasing the HOTS of students, teachers need to consider the use of forms of Assessment, not only paying attention to the application of learning. The selection of an appropriate learning model must be accompanied by the selection of the right form of Assessment so that learning becomes effective and efficient. Carrying out further research in developing the results of this research needs to be done by involving more varied alternative forms of Assessment. In addition, it can be done on other mathematical materials so that more comprehensive results are obtained. REFERENCE ADDIN Mendeley Bibliography CSL_BIBLIOGRAPHY Ames, C. (1992). Classroom: Goals, structures, and student motivation. Journal of Educational Psychology. Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing. Longman. Banning, M. (2008). The think-aloud approach as an educational tool to develop and assess clinical reasoning in undergraduate students. Durning, S. J., Artino, A. J., Beckman, T. J., Graner, J., van der Vleuten, C., Holmboe, E., & Schuwirth, L. (2013). Does the think-aloud protocol reflect thinking? Exploring functional neuroimaging differences with thinking (answering multiple choice questions) versus thinking aloud. Ferreri, S. P., & O'Connor, S. K. (2013). Redesign of a large lecture course into a small group learning course. Hammonds, D. S., & Schwarze, M. J. (2019). Toward oral Assessment in counselor education: Practice-based measurement of competence in pre-practicum experiences. Hounsell, D., N. Falchikov, J. Hounsell, M. Klampfleitner, M. Huxham, K. Thompson, and S. B. (2007). Innovative Assessment across the disciplines: An analytical review of the literature. Huxham, M., Campbell, F., & Westwood, J. (2012). Oral versus written assessments: A test of student performance and attitudes. Joughin, G. (1998). Dimensions of oral Assessment. Assessment & Evaluation in Higher Education. Krulik, S., & R. J. A. (1999). Innovative Task to Improve Critical and Creative Thinking Skill. In Stiff, Lee V. & Curcio, Frances R.(Eds). Levett-Jones, T., Gersbach, J., Arthur, C., & Roche, J. (2011). Implementing a clinical competency assessment model that promotes critical reflection and ensures nursing graduates' readiness for professional practice. Minton, C. A. B., Gibson, D. M., & Morris, C. A. W. (2016). Evaluating student learning outcomes in counselor education. John Wiley & Sons. Park, M. S. (2020). Rater Effects on L2 Oral Assessment: Focusing on Accent Familiarity of L2 Teachers. 14 |JISAE (Journal of Indonesian Student Assessment and Evaluation) |Volume 9 Number 1 Resnick, L. B. (1992). Education and learning to think. Washington DC: National Academy Press. Richardson, S., Mishuris, R., O'Connell, A., Feldstein, D., Hess, R., Smith, P., & Mann, D. (2017). Research paper: "Think-aloud" and "near live" usability testing of two complex clinical decision support tools. Sadler, D. R. (2009). Indeterminacy in the use of preset criteria for Assessment and grading. Sugiyono. (2019). Statistika untuk Penelitian. Bandung. Swank, J. M. (2012). Using games to creatively enhance the counselor education curriculum. Tekian, A., & Yudowsky, R. (2009). Oral examinations. In S. M. Downing & R. Yudkowsky (Eds.). Turner, M., & Davila-Ross, M. (2015). Using oral exams to assess psychological literacy: The final year research project interview.