This study is intended to understand teaching quality of English student teachers when they conduct their teaching practicum. Teaching quality is conceptualized based on the principles of effective teaching resulted by teacher effectiveness studies. Thes IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 192 Development of an Authentic Assessment Model in Mathematics Learning: A Science, Technology, Engineering, and Mathematics (STEM) Approach I WAYAN WIDANA 1 , AGUS TATANG SOPANDI 2 , AND AGUS TATANG SOPANDI 3 Abstract STEM (Science, Technology, Engineering, Mathematics) approach in mathematics learning is very suitable for improving higher-order thinking skills. However, the ability of teachers to develop authentic assessment models based on the STEM approach is still low. This study aims to develop an authentic assessment model suitable for learning based on the STEM approach in mathematics at Senior High School. This study used a Research & Development design adapted from the Borg & Gall model. Data were collected using a validation sheet and a questionnaire. The data source was 269 teacher respondents selected using multistage random sampling technique, while two experts from higher education as validators and 21 senior high school math teachers experienced as panellists were determined by purposive sampling technique. Data were analysed through a quantitative approach. The product of this research is an authentic assessment model consisting of twenty project tasks. The results of the analysis show that 16 project tasks with very good quality and four project tasks with good quality, declared appropriate and in accordance with the STEM approach-based learning. Keywords Authentic assessment, STEM-based learning approach, mathematics 1. Mathematics Education Study Program, Faculty of Teacher Training and Education, PGRI Mahadewa University, Indonesia, Denpasar, Bali, Indonesia; i.wayan.widana.bali@gmail.com 2. Elementary School Teacher Education Study Program, Teacher Training and Education Faculty, Universitas Terbuka; UPBJJ Denpasar, Bali; atatang@ecampus.ut.ac.id 3. Statistics Study Program, Faculty of Science and Technology, Universitas Terbuka; UPBJJ Denpasar, Bali; isuwardika@ecampus.ut.ac.id mailto:i.wayan.widana.bali@gmail.com mailto:atatang@ecampus.ut.ac.id mailto:isuwardika@ecampus.ut.ac.id ASUS Rectangle ASUS Rectangle ASUS Rectangle ASUS Typewriter GEDE SUWARDIKA3 IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 193 Introduction One of the most dominant features in the Industrial Revolution 4.0 is the rapid use of information and communication technology in various aspects of life, including education (Morrar et al., 2017). The development of science and technology is increasingly sophisticated and integrated with social sciences and humanities. This progress has resulted in geographic barriers of space and time, which have been the determinants of the speed and success of human mastery of science, which can be done easily and quickly (Andersson & Mattsson, 2015). Everyone can easily interact, communicate, and transact anytime and from and wherever they are. Thus, mastery of information technology is an important issue to do through education (Dutton, 2014). The 21st-century education paradigm can be formulated as follows (Lee et al., 2014; Marolt et al., 2015): (1) education must be oriented towards mathematics, science, social sciences, and humanity; (2) science not only makes a person knowledgeable but also adopts a scientific attitude, namely critical, logical, analytical, innovative, and creative with adaptability; (3) At every level of education, it is necessary to instill a spirit of independence because individual independence underlies the independence of the nation. Thus, for the Indonesian nation to have high competitiveness in the association of the international community, students should be equipped with some competencies needed in the era of globalization, namely: creativity, critical thinking, communication, and collaboration (Widana, 2020). Mathematics trains the ability to think logically, analytically, systematically, critically, innovatively, and creatively, as well as the ability to work together (Sudiarta & Widana, 2019). Mathematics is used in various aspects of life and other scientific disciplines. The use of mathematics as a means of communication-based on symbols, which are brief and clear, can present information in various ways and develop creativity (Isnawan & Wicaksono, 2020). Mathematical skills are part of life skills that students must have, especially in developing reasoning, communication, and solving problems faced in everyday life. Learning mathematics cannot be separated from the context of life and civilization (Widana et al., 2019). The mathematics learning process should be carried out through interactive, inspirational, fun, challenging, and motivating students to participate actively and provide sufficient space for initiative, creativity, and independence according to students’ talents, interests, and physical and psychological development. The teacher tries to inspire students with challenging and fun mathematical ideas packaged in interactive mathematics learning (Kheruniah, 2013). The space for the concept discovery process for students allows the development of student initiative and creativity. Therefore, teachers must be creative in choosing the learning approach used by the characteristics of the subject matter. The learning approach should also consider integrating information technology users and promoting the development of higher-order thinking skills. According to Sun and Wu (2016), project-based mathematics learning can increase student participation in learning and increase creativity to solve problems. Project-based learning is designed to solve complex problems through investigative activities and develop real problem-solving skills. Projects/activities presented in form of learning are a medium for students to explore, assess, interpret, synthesize, and draw conclusions to produce various learning outcomes IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 194 (Richard, 2018). Activities carried out to complete projects can improve higher-order thinking skills because students are trained to think from planning, project implementation, analysis, evaluation, and developing ideas creatively. The achievement of learning objectives should be measured using an assessment instrument according to the learning approach used. The coherence between the learning approach and the assessment model used is very important for teachers to pay attention to. A good assessment instrument should describe the achievement of learning outcomes objectively and can be used as material for reflection to improve the learning process (Weinstein & Preiss, 2017). Assessment of learning outcomes can help teachers improve the quality of learning and provide feedback for students to improve learning outcomes. Thus, if the teacher uses a project-based learning approach, the assessment instrument will also use a project-based assessment model. Theoretical Framework STEM approach in learning mathematics STEM-based learning uses science, technology, engineering, and mathematics approaches in real contexts that connect schools, the world of work, and the global world so that students can compete in the industrial revolution 4.0. Wang & Degol (2017) state that to determine learning objectives using the STEM framework, it is necessary to conduct an assessment using an authentic assessment model according to the characteristics of STEM-based learning. The assessment model can be carried out in two ways, namely: (a) assessment for learning, namely assessment during the learning process (on-going), which aims to improve the learning process, diagnose learning difficulties, and determine the development of student competency achievements, and (b) assessment of learning, namely an assessment carried out at the end of the lesson to determine the achievement of student competencies. The STEM approach in mathematics learning is very suitable for developing higher-order thinking skills and improving students’ ability to solve problems based on contextual cases (Galloway et al., 2013). STEM-based learning can connect the knowledge learned in schools, the world of work, and the global world so that students can compete in the industrial revolution 4.0. Two main aspects characterize STEM learning: the science and engineering design processes (EDP), both closely related to supporting learning. The scientific process is a tiered process consisting of 5 main stages, namely: (1) asking questions or making observations; (2) compiling hypothesis; (3) compile an estimated answer; (4) conducting tests/experiments; and (5) finding and suggesting conclusions (Stout et al., 2011). While EDP is a cycle stage: (1) mapping the problem, (2) designing solutions for problem-solving, (3) modeling aims to prove that problem solving is possible, (4) testing the model and the results will be evaluated whether the solution model is solving the problem is already effective in solving the problem or not, if it is deemed ineffective, an improvement in the design of the problem-solving model is carried out (Roberts, 2012). Models introduced in the design of engineering processes can be in products, processes or systems. The relationship between scientific processes and engineering process design can be explained in Figure 1 below. IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 195 Figure 1. Relationship between science process and EDP Scientific process Engineering Design Process  Ask Question  Observe  Experiment  Measure Investigating Evaluating  Imagine  Reason  Calculate  Prediction Developing, Explanations, and Solutions In the picture on the left: the dominant activity is a scientific process with an observation, inquiry, and experiment approach based on phenomena and problems in the real world. The results of these observations can be linked to EDP (image at right) through the analysis process. This process is the first stage of EDP in problem mapping, which is carried out with a scientific process to provide a comprehensive picture of the problem. Analysis of the results of observations of problems will be solved using theory and modeling that arise from finding solutions, critical thinking, and creative thinking, which are predominantly carried out with EDP (National Academy of Sciences, 2011). In the next stage, the science process and EDP are needed to analyze whether the theory and model proposed can solve problems by collecting, testing, and analyzing problem-solving solutions for later evaluation and refinement. In the three parts in the picture, analysis is the key part of linking science and EDP. Scientists and engineers will work together to do the best problem solving with all available resources (Breiner et al., 2012). To solve problems, the two parts in the picture analyze the problem, and the data is easier to describe through modeling, including using sketches, diagrams, mathematical relationships, simulations, and prototypes to ensure that the solution can solve the problem at hand. The use of these models requires good mathematical skills. This problem-solving pattern is introduced to students through STEM-based learning. REAL WORD COLLECT DATA, TEST SOLUTIONS THEORIES AND MODELS FORMULATE, HYPOTHESES, PROPOSED SOLUTIONS ANALYZE IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 196 Authentic assessment Assessment is an integral part of the learning process, so the assessment model used must be in accordance with the learning approach used in teaching in the classroom. The assessment model used to test the achievement of learning objectives must be in line with the learning process that has been carried out (Manurung & Aryni, 2019). To determine the achievement of learning objectives using the STEM framework, it is necessary to evaluate using an authentic assessment model according to the characteristics of STEM-based learning. The assessment model can be carried out in two ways: assessment for learning carried out during the learning process (on-going), which aims to improve the learning process, diagnose learning difficulties, and determine student competency achievements. While the assessment of learning is carried out at the end of the lesson to determine the achievement of student competencies. Authentic assessment is an assessment carried out comprehensively to assess starting from input, process, and learning output (Suarimbawa et al., 2017). Authentic assessment collects information by the teacher about the development and achievement of learning through various techniques that can reveal, prove, or demonstrate precisely that the learning objectives have been truly mastered and achieved. Some of the characteristics of an authentic assessment are as follows (Fatonah et al., 2013): (1) is a comprehensive part of the learning process; (2) reflects the results of the learning process in real life, not only based on existing conditions in school; (3) using a variety of instruments, measurements, and methods under the characteristics and essence of the learning experience; (4) is comprehensive and holistic covering all aspects of attitudes, knowledge, and skills; and (5) includes an assessment of the learning process and learning outcomes. An authentic assessment model based on the STEM approach The authentic assessment model based on the STEM approach is a project-based assessment model that can comprehensively assess the cognitive, affective, and psychomotor domains. This assessment model integrates the components of the STEM approach, including science, technology, engineering and mathematics (Sagala et al., 2019). Students’ attainment of attitudinal competence is carried out through observations of students’ attitudes (affective) in carrying out given project tasks or the results of implementing given project assignments. Assessment of competence in the cognitive realm is carried out through problem-solving abilities, including analyzing problems, evaluating, and designing problem-solving using a mathematical model based on information technology. Meanwhile, assessing the psychomotor domain is carried out to assess students’ skills in completing the project in its entirety, starting from planning, implementation, reports, and presentation skills. An authentic assessment model based on the STEM approach is packaged in project tasks that can be done individually or in groups. Arikunto (2013) states that authentic assessment based on project tasks is an assessment that measures the competence of knowledge, attitudes, and skills towards an investigative process to find meaningful benefits IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 197 for human life that must be completed within a certain time. The assessment criteria use a rubric. A guideline is used to assess a certain set of activities in project tasks (Ekawarna et al., 2020). Rubrics are needed to minimize the subjectivity of the assessment, so the availability of rubrics is essential in project-based assessments. Methodology Research design, respondents, and locale of the study This study used a Research & Development design adapted from the Borg & Gall (2003) model. The Borg and Gall design consist of ten stages, including 1) preliminary study and preliminary information gathering, 2) planning, 3) product design development, 4) product design validation, 5) initial product revision, 6) limited scope field testing, 7) further product revisions, 8) extensive field testing, 9) final product revisions, and 10) product finalization and dissemination. This research is only focused on the implementation of the seven development stages of the Borg and Gall design, starting from the stages: 1) preliminary study and information gathering, 2) planning, 3) product design development, 4) product design validation, 5) initial product revision, 6) limited scope field test, 7) further product revision. The reason is, this research was only conducted for one year, and at the time this research was carried out, it was still in the conditions of the Covid-19 pandemic so that extensive trials involving high school students could not be carried out. The preliminary study stage and initial information collection involved 269 high school math teachers selected from 15 provinces throughout Indonesia. The sample selection used a multistage random sampling technique. Preliminary studies and preliminary information collection are focused on obtaining data on the understanding of high school mathematics teachers on learning mathematics using the STEM approach and developing authentic assessments of learning based on the STEM approach. The validation activity of the product design of the authentic assessment model of learning based on the STEM approach aims to test the feasibility of the authentic assessment model that has been developed in terms of the material aspect, the suitability of the authentic assessment with the learning approach, as well as the quality of project tasks. The validation of the authentic assessment model was carried out by two experts selected from universities. The limited scope field test phase involved 21 high school math teachers as panelists selected based on academic competence, teaching experience using the STEM approach, and experience developing assessments at both the provincial and national levels. Activities focused on analyzing the quality and feasibility of authentic assessment models on learning based on the STEM approach. Data collection and analysis In this study, there were three types of data collected in accordance with the focus on each stage of the research. Considering that this research was conducted during the Covid-19 pandemic and following health protocols, all data collection instruments were IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 198 distributed to respondents online using a google form. Likewise, the collection of research data is carried out online. Data at the preliminary study stage and information collection were collected using a questionnaire consisting of 20 questions/statements using a Likert scale. The validity test of the questionnaire was carried out using the Pearson Product Moment formula, while the reliability test of the questionnaire used the Cronbach Alpha formula. Analysis of the validity and reliability of the questionnaire was conducted using the SPSS 23.0 for Windows program. Furthermore, the data obtained were analyzed descriptively quantitatively. The data from the validation of the authentic assessment model of STEM-based learning by two experts from universities were collected using a validated instrument developed by the researcher referring to the characteristics of authentic assessment and STEM learning approaches. Furthermore, the data from the validation results of the assessment model by experts were analyzed using the Gregory formula (2000). Limited trial data for authentic assessment models of learning based on the STEM approach were collected using an assessment sheet of the assessment model, which was prepared to refer to the characteristics of authentic assessment of learning based on the STEM approach, including material aspects, construction, language, and elements of the STEM approach. Limited trial data were analyzed by panelists using the Thurstone 11 scale, based on the value of Mean (Me), Median (Md), Mode (Mo), and interquartile range (Q3-Q1) as follows: [1] the higher the Me value, shows that the items have better quality of; [2] the higher the Md value, the better the quality of the questions; [3] the value of Mo shows the tendency of the results of the assessment of each panelist on the quality of the items; and [4] the smaller the value of the interquartile range (Q3-Q1), indicating that the stronger the panel agreement is (Dja’ali & Muljono, 2008). Ethical considerations The data from this study were fully obtained from participants who had volunteered to contribute to this study. To protect participants’ privacy, their names and agency addresses are not explicitly written in this article. This was done as a form of respect to all participants for their honesty and sincerity. Findings Preliminary study and information gathering Data at the preliminary study stage and initial information collection were collected using a questionnaire consisting of three main dimensions, namely (a) teachers’ understanding of the concept of learning based on the STEM approach, (b) teachers’ understanding of the characteristics of authentic assessments, and (c) teachers’ skills in preparing authentic assessments. On learning based on the STEM approach. Each of these dimensions is translated into several indicators presented in the form of questions/statements. The complete research results on each of the main dimensions can be seen in tables 1, 2, and 3 below. In statement number 1 of the table above, cumulatively as IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 199 many as 49.81% of mathematics teachers chose responses 4 and 5, which means that the teachers understand the basic concepts of learning mathematics based on the STEM approach, 35.21% of the teachers expressed doubt, and 14.98% the teacher said he did not understand at all. Likewise, in statement number 2, the teacher chose responses 4 and 5 cumulatively of 46.07%, which means that theoretically, high school mathematics teachers have understood the importance of the STEM approach in mathematics learning, 40.07% of teachers stated that it was normal, and 13.86% said it was not important. According to Bloom’s Taxonomy, the teacher’s understanding of operational verbs is good. This can be seen in the statement/question indicator number 3. It can be seen that cumulatively the teacher chooses the 4 and 5 responses as much as 75.28%, 19.48% of the teachers expressed doubt, and 5.24% of the teachers stated they did not understand. Table 1. Teachers' understanding of learning based on the STEM approach Statement 1 2 3 4 5 6 Response 1 (%) 2.25 1.50 1.12 0.75 7.49 12.73 Response 2 (%) 12.73 12.36 4.12 3.75 27.34 36.33 Response 3 (%) 35.21 40.07 19.48 25.47 34.46 26.59 Response 4 (%) 41.20 38.95 53.18 54.31 22.10 20.97 Response 5 (%) 8.61 7.12 22.10 15.73 8.61 3.37 Total (%) 100 100 100 100 100 100 In statement number 4, the cumulative number of teachers who choose responses 4 and 5 is 70.04%, which means that most teachers have understood the operational verb grouping according to Bloom’s Taxonomy. 25.47% of the teachers expressed doubts, and the remaining 4.50% of the teachers stated that they did not understand. However, in statements 5 and 6, the opposite condition occurs in statement number 5 regarding the teacher’s understanding of the characteristics of basic competencies presented using the STEM approach. As many as 30.71% of the teachers chose responses 4 and 5, the teachers understood. Meanwhile, 34.46% expressed doubt, and 34.83% said they did not understand. In statement number 6, cumulatively, 24.34% of teachers who chose responses 4 and 5 stated that they were ready to carry out STEM-based learning, 26.59% said they were unsure, and 49.06% said they were not ready. Table 2. Teachers' understanding of the characteristics of authentic assessment Statement 7 8 9 10 11 Response 1 (%) 5.24 2.25 4.12 4.49 0.37 Response 2 (%) 32.58 21.72 11.24 17.23 4.12 Response 3 (%) 42.32 40.45 23.22 38.58 13.86 Response 4 (%) 17.60 28.84 48.69 30.34 41.95 Response 5 (%) 2.25 6.74 12.73 9.36 39.70 Total (%) 100 100 100 100 100 IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 200 Table 2 in question/statement number 7 is a negative statement, indicating that 37.82% (responses 1 and 2) teachers really need models or examples of authentic assessment, and 42.32% say they do. Only 19.85% (responses 4 and 5) of high school mathematics teachers stated that they did not need authentic assessment models or examples. This is reinforced by statement number 11, which states that only 0.37% of teachers do not need examples or authentic assessment models of learning based on the STEM approach. On the other hand, 99.63% of the teachers stated that they even need 39.70% of them really need authentic models or examples of learning based on the STEM approach. In statement number 8, as many as 35.58% (responses 4 and 5), the teachers stated that they could not distinguish between authentic assessments or not, 40.45% expressed doubts, and 23.97% stated that they were able to distinguish between authentic assessments or not. Statement number 9 states 61.42% (responses 4 and 5) teachers need a guide/literature to write authentic assessments on learning based on the STEM approach, 23.22% say it is necessary, and 15.36% say it is not necessary. Statement number 10, as many as 39.70% of teachers stated that they did not understand the characteristics of authentic assessment based on the STEM approach, 38.58% expressed doubt, and 21.72% stated that they understood. Table 3. Teachers' skills in preparing authentic assessments Statement 12 13 14 15 16 17 18 19 20 Response 1 (%) 0.00 1.12 1.50 2.62 5.24 1.12 1.12 3.75 3.75 Response 2 (%) 6.37 13.48 12.73 17.60 30.34 6.74 15.36 20.60 22.10 Response 3 (%) 28.09 38.58 41.95 36.70 34.83 32.96 40.45 39.33 36.70 Response 4 (%) 46.44 35.96 33.71 32.96 25.47 45.32 32.58 26.97 26.97 Response 5 (%) 19.10 10.86 10.11 10.11 4.12 13.86 10.49 9.36 10.49 Total (%) 100 100 100 100 100 100 100 100 100 In statement number 12, table 3 above shows that teachers have not developed project-based question indicators according to the demands of basic competencies. Cumulatively as much as 65.54% (responses 4 and 5) stated that the teachers had not been able to develop project-based question indicators, 28.09% (response 3) stated that teachers were still in doubt whether the indicators compiled were correct or not, and 6.37% of other teachers stated that they were able to prepare project-based question indicators according to basic competencies. Whereas in statement number 13, a cumulative amount of 46.82% (responses 4 and 5) states that teachers have not been able to develop authentic assessments that fully measure the domains of knowledge, attitudes, and skills, 38.58% of teachers stated that they did not know whether the assessment was prepared an authentic assessment or not, and 14.6% (responses 1 and 2) stated that they were able to develop authentic assessments. In statement number 14, a cumulative amount of 43.82% (responses 4 and 5), the teacher stated that they were very motivated to learn to develop an authentic project-based assessment model in accordance with the STEM approach based learning because of moral IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 201 responsibility, 41.95% stated that they were motivated to learn to develop authentic assessments because it was teacher obligations, and 14.23% stated that they were not motivated to develop authentic assessments because it was difficult to compile them. Furthermore, statement number 15 cumulatively of 43.07% states that high school mathematics teachers do not compile the question grids. The teachers immediately write the question items on the test or exam question manuscript without writing the grid first. Only 20.22% (responses 1 and 2) of the teachers arranged the question grid before writing the question cards, while 36.70% of the teachers stated that sometimes. Statement number 16 states that cumulatively, as many as 29.59% (responses 4 and 5) teachers conducted a qualitative analysis of the assessment before it was tested, 34.83% stated that sometimes they did the qualitative analysis, for example, during school exams or final semester tests only, and 35.58% stated no have carried out a qualitative analysis of the assessment before being tested. Statement number 17 states that cumulatively 59.18% (responses 4 and 5) teachers are not yet skilled at compiling activity/practice-based questions, 32.96% of teachers say that they have tried to compile activity/practice-based questions, and 7.86% of teachers say they are used to developing activity-based questions. In statement number 18, the teachers (cumulatively 43.07% for responses 4 and 5) stated that they still had difficulty preparing assessments based on contextual problems, 40.45% of teachers stated that they had tried but were not skilled, and 16.48% of teachers stated that they were used to developing contextual problem-based questions. In addition, statement number 19 cumulatively amounted to 36.33% stated that teachers still had difficulty developing information technology-based assessments, 39.33% of teachers stated that they were not used to it, and the remaining 24.35% of teachers were used to developing information technology-based assessments. Statement number 20 (negative statement) states that mathematics teachers do not understand the mechanism for preparing authentic assessments based on STEM-based learning well as much as 37.46% (responses 4 and 5), as many as 36.70% are still unsure, and the remaining 25.85% teachers state already understand the mechanism of preparing authentic learning assessments based on the STEM approach. Based on the data analysis above, the following findings were obtained: (a) the understanding of high school mathematics teachers of the basic concepts and characteristics of authentic assessment of learning based on the STEM approach is theoretically quite good, (b) teachers’ understanding of the importance of learning mathematics based on the approach STEM is good, but it is not accompanied by the teacher’s ability to compile an authentic assessment of learning based on the STEM approach, (c) in the preparation of the assessment, most high school math teachers have not compiled a grid and conducted a qualitative item analysis This means that the assessment preparation mechanism is not in accordance with the proper mechanism, and (d) high school mathematics teachers need an authentic assessment model for learning based on the STEM approach. Therefore, it is necessary and urgent to develop an authentic assessment model for learning based on the STEM approach. IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 202 Planning At the planning stage, several activities were carried out as follows: (a) analyzing basic competencies (KD) that could be presented with STEM-based learning; (b) formulating question indicators as a reference for writing questions or project assignments, and (c) compile an authentic assessment grid based on the STEM approach. Another activity undertaken is looking for contextual cases related to basic competencies for authentic assessment models via the internet and other sources. Product design development The product developed is an authentic assessment model based on the STEM approach in high school mathematics subjects. The development of the initial product design was carried out by designing an authentic assessment model based on the STEM approach by referring to the arranged grid. In this study, 20 authentic assessment models based on the STEM approach were developed in high school mathematics subjects. Product design validation Two university experts carried out the initial product design validation. Validation is carried out using a validation instrument containing the results of the assessment of each expert, notes/suggestions for each evaluation, and conclusions on the appropriateness of the assessment instrument developed. To determine the content validity coefficient, it can be calculated using the Gregory (2000) formula. B Remarks: A : The cell that shows the disagreement between the two experts B and C : The ells that show different views between the first expert and the second expert (the first expert agrees/is very relevant and the second expert disagrees/is less relevant and vice versa). D : The cell that shows valid agreement between the two experts. Table 4. Content validity Content Validity Coefficient Content Validity Criteria 0,80 – 1,00 Very high 0,60 – 0,79 High 0,40 – 0,59 Normal 0,20 – 0,39 Low 0,00 – 0,19 Very low IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 203 Table 5. The results of the validation of authentic assessment models by experts Expert 1 Less relevant Relevant Expert 2 Less relevant 1 10, 13 Relevant 9, 11 2, 3, 4, 5, 6, 7, 8, 12, 14, 15, 16, 17, 18, 19, 20 In accordance with table 5 above, the value of the validity of the contents of the authentic assessment model for learning based on the STEM approach can be calculated as the High category. Some important notes given by experts include: (1) for project model number 1, the rubric needs to be made more explicit so that the objectivity of the assessment could be appropriate. Attitude assessment indicators should be made more operational so that the assessment can be carried out more accurately and objectively; (2) in project model number 9, students cannot factorize the problems given in accordance with predetermined learning objectives, the assessment must be revised. The problems that are presented as stimuli in the project are less contextual; (3) in project model number 10, if the concept of other subjects is applied to mathematics, it is advisable to provide a brief explanation of the concept to deliver better student understanding; (4) in project model number 11, the project is too complicated for students to complete, it needs collaboration with other subjects; (5) in the project model number 13, the implementation instructions need to be clarified regarding the implementation period, the location of the work, and the instructions for writing the report; (6) for other project authentic assessment models to be improved editorial refers to the PUEBI writing guidelines. Early-stage product revision The revision of an authentic assessment model for learning based on the STEM approach was carried out based on input and important notes provided by experts. Improvements were made to editorial refinement, layout rearrangement, and adjustment of the stimulus by changing the stimulus to be more contextual and carry novelty. Each authentic assessment model is given a special note, refined by the research team while still paying attention to basic competencies supported by other scientific concepts. Limited scope field test After completing the revision of the initial product in the form of a learning assessment model based on the STEM approach, a limited trial was carried out involving 21 high school math teachers as panelists. The results of the limited scope field test assessment are presented in table 10 below. IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 204 Table 6. Panelist assessment results No. Mean Modus Q1 Q2 Q3 Q3-Q1 Quality and interpretation 1 9.88 9.83 9.78 9.89 10.17 0.39 Good, decent enough, relevant, and very strong panelist approval 2 10.21 10.33 9.83 10.33 10.61 0.78 Very good, appropriate, relevant, and very high panelist approval 3 10.24 10.39 9.94 10.39 10.56 0.61 Very good, appropriate, relevant, and very high panelist approval 4 10.32 10.61 10.00 10.44 10.61 0.61 Very good, appropriate, relevant, and very high panelist approval 5 10.26 10.22 10.06 10.28 10.67 0.61 Very good, appropriate, relevant, and very high panelist approval 6 10.46 10.50 10.39 10.50 10.78 0.39 Very good, appropriate, relevant, and very high panelist approval 7 10.30 10.11 10.06 10.28 10.72 0.67 Very good, appropriate, relevant, and very high panelist approval 8 10.25 10.22 10.11 10.28 10.56 0.44 Very good, appropriate, relevant, and very high panelist approval 9 9.88 9.50 9.50 9.83 10.22 0.72 Good, decent enough, relevant, and very strong panelist approval 10 10.06 10.06 9.83 10.06 10.33 0.50 Very good, appropriate, relevant, and very high panelist approval 11 10.05 10.06 9.83 10.06 10.33 0.50 Very good, appropriate, relevant, and very high panelist approval 12 10.41 10.56 10.17 10.53 10.61 0.44 Very good, appropriate, relevant, and very high panelist approval 13 10.02 10.00 9.78 10.00 10.28 0.50 Very good, appropriate, relevant, and very high panelist approval 14 9.93 9.78 9.67 9.83 10.11 0.44 Good, decent enough, relevant, and very strong panelist approval 15 10.29 10.17 10.11 10.22 10.61 0.50 Very good, appropriate, relevant, and very high panelist approval 16 10.32 10.44 10.00 10.44 10.50 0.50 Very good, appropriate, relevant, and very high panelist approval 17 10.31 10.44 9.94 10.44 10.56 0.61 Very good, appropriate, relevant, and very high panelist approval 18 10.49 10.50 10.44 10.50 10.67 0.22 Very good, appropriate, relevant, and very high panelist approval 19 10.13 10.17 9.83 10.17 10.39 0.56 Very good, appropriate, relevant, and very high panelist approval 20 10.07 9.83 9.83 9.89 10.17 0.33 Good, appropriate, relevant, and very high panelist approval The data in table 6 above show that the panelists assessed the quality of the authentic assessment model for learning based on the STEM approach as follows: (1) as many as 16 authentic assessment models expressed in very good quality, this can be seen from the mean value above 10 or close to the maximum score of 11 according to the Thurstone scale, the IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 205 median value (Q2) is also above 10, the mode value is above 10, and the panelist agreement is also very strong; (2) as many as 4 authentic assessment models are stated in good quality, this can be seen from the mean value above 9 on the 11 Thurstone scale, the median value (Q2) is also above 9, the mode value is above 9, and the panelists’ agreement is also very strong. ; (3) the panelists suggested that the indicators be refined to be more specific in measuring the achievement of basic competencies; and (4) to make improvements to the language elements so that students can understand them more easily. Discussion The findings in the preliminary study indicate that high school mathematics teachers need to be assisted by examples or authentic assessment models of learning based on the STEM approach. Although, in theory, some of them already understand the basic concepts of STEM learning, in practice, the teachers have not been able to develop an authentic assessment model for learning based on the STEM approach. Many factors affect the ability of teachers to develop an authentic assessment of learning based on the STEM approach, including (Bertschy et al., 2013): (a) creativity, teachers who have low creativity tend to just wait for examples and do not have ideas to try to develop. teacher’s creativity is very important to be developed through periodic exercises; (b) mastery of information technology is currently one of the mandatory competencies for teachers. What’s more, the teacher will develop an assessment based on the STEM approach. One of the elements of STEM is a technology that demands the ability of teachers to learn to use information technology in learning and assessment. Teachers who master information technology will be richer in knowledge, especially getting stimuli downloaded via the internet. So that mastery of information technology becomes mandatory for teachers at this time; (c) high work motivation, highly motivated teachers will do anything related to the teacher’s ability to develop authentic assessments based on the STEM approach. They have a high responsibility, a genuine commitment to producing authentic assessments based on the STEM approach. The authentic assessment model is not entirely new for teachers. It has been discussed for a long time in various meetings. However, in practice, it is still rare for teachers to try to develop authentic assessments. Theoretically, authentic assessment is an assessment of collecting information by teachers about the development and achievement of learning through various techniques that can reveal, prove or demonstrate precisely that learning objectives have been truly mastered and achieved (Habibi, 2015). Some of the characteristics of an authentic assessment are as follows: (1) assessment is a comprehensive part of the learning process; (2) the assessment reflects the results of the learning process in real life, not only based on conditions in school; (3) using a variety of instruments, measurements, and methods in accordance with the characteristics and essence of the learning experience; (4) comprehensive and holistic assessment covering all aspects of attitudes, knowledge, and skills; and (5) assessment includes an assessment of the learning process and learning outcomes. Authentic assessment of STEM-based learning must reflect the following characteristics: (1) assignment design must reflect what activities students have to do; (2) the IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 206 physical context, in which the assessment must be carried out so that it reflects how to measure attitudes, knowledge, and skills; (3) social context, with regard to social interactions according to the real world with whom to do it; (4) what products are produced, the assessment must be able to reveal the students’ ability to demonstrate, present, or communicate with others; (5) what criteria and standards should be evaluated. Some of the advantages of authentic assessment include (1) authentic assessment oriented to the assessment of the learning process, thus through authentic assessment, the teacher will be able to find out where the strengths and weaknesses of students are; (2) authentic assessment can describe the achievement of a student in learning in the form of learning gain or progress, not just indicated by the numbers stated on the report card; (3) more authentic assessment and results will improve the teaching and learning process, students know more clearly their obligations to master the assigned tasks, and teachers believe that the results of the assessment are meaningful and useful for improving teaching; (4) the curriculum does not merely increase student knowledge, but competence as a whole that reflects knowledge, skills, and attitudes according to the characteristics of each subject. Some of the limitations of authentic assessment can be explained as follows. The obstacles to carrying out authentic assessments on knowledge competencies, namely: (1) developing authentic assessment instruments is not easy for most teachers, and (2) it takes a long time to check and assess student work. Constraints in the authentic assessment of skills competencies, namely: (1) not all students are active in class project activities and rely more on clever children to complete assignments; (2) students have not yet seriously worked on the project; (3) project assignments are often seen as difficult for students; (4) inadequate practice tools, making it difficult to measure individual student skills; (5) sometimes requires a quite expensive fee. The teacher should bring an authentic assessment sheet during the assessment to be completed immediately. Meanwhile, the limitations of implementing authentic assessments on attitude competency assessments include: (1) the required assessment time is quite long; (2) no honesty indicator can describe honesty aspects objectively, and (3) difficulty in monitoring the limited behavior of students in the school environment. Conclusion and Recommendations Based on the results of the analysis and interpretation that have been carried out previously, it can be stated that the authentic assessment model of the STEM approach-based learning developed in this study has good quality as many as four project tasks and 16 other project tasks are of very good quality. The project assignment cards that have been developed have been revised several times so that what is judged by the panelists is the final product. Before assessing the panelists, the project assignment cards were assessed by experts related to the material aspects, construction, language, STEM elements, and authentic assessment characteristics. However, in the assessment by the panelists, there were still some weaknesses that should be taken into consideration for the next revision. The process of refining the model has been carried out many times in accordance with the Research & Development procedure. The products produced in this study are expected to inspire teachers to develop authentic assessments based on the STEM approach in high IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 207 school mathematics subjects. Teachers should creatively develop authentic assessment models to improve the function and objectives of the assessment so that the assessment results can be used to improve the learning process and as a measuring tool to determine the achievement of learning objectives. References Andersson, P., & Mattsson, G. L. (2015). Service innovations enabled by the internet of things. IMP Journal, 9(1), 85–106, http://doi.org/10.1108/IMP-01-2015-0002. Arikunto, S. (2013). Dasar-dasar evaluasi pendidikan. Jakarta: Bumi Aksara. Bertschy, F., Künzli, C., & Lehmann, M. (2013). Teachers’ competencies for the implementation of educational offers in the field of education for sustainable development. Sustainability (Switzerland), 5(12), 5067–5080, https://doi.org/10.3390/su5125067. Borg, W.R and Gall, M.D. (2003). Educational research: An introduction 4 th edition. London: Longman Inc. Breiner, J., Harkness, S., Johnson, C., & Koehler, C. (2012). What is STEM? A discussion about conceptions of STEM in education and partnerships. School Science and Mathematics, 112 (1), 3-11. ja’ali, H. & Muljono, P. (2008). Pengukuran dalam bidang pendidikan [Measurement in Education]. Jakarta: PT. Grasindo. Dutton, H. W. (2014). Putting things to work: Social and policy challenges for the internet of things. Info, 16(3), 1–21, https://doi.org/10.1108/info-09-2013-0047. Ekawarna, Yusdi, A., Qurotta, A., Romios, S., & Tarnak. (2020). Analyzing the effect of need for achievement and locus of control on student entrepreneurial intentions. Indonesian Research Journal in Education, 4(1), 28-42. Fatonah,S., Suyata, P., & Prasetyo, Z. K. (2013). Developing an authentic assessment model in elementary school science teaching. Journal of Education and Practice, 4(13), 50-61. Galloway, L. M., Pease, J. L., & Rauh, A. E. (2013). Introduction to altmetrics for science, technology, engineering, and mathematics (STEM). Librarians, Science & Technology Libraries, 32(4), 335-345, DOI: 10.1080/0194262X.2013.829762. Gregory, J. R. (2000). Psychological testing history, principles, and applications. Boston: Allyn and Bacon. Habibi, B. (2015). The Influence of principal managerial competence and work motivation on teacher professionalism of vocational high schools. Dinamika Pendidikan, 10(2), 119–124, https://doi.org/10.15294/dp.v10i2.5104. Isnawan, M. G., & Wicaksono, A. (2020). Students mathematics learning achievement from mathematics teacher performance and principal managerial competencies point of view. Indonesian Journal of Mathematics Education, 3(2), 76-86, DOI: 10.31002/ijome.v3i2.3375. Kheruniah, A. E. (2013). A teacher personality competence contribution to a student study motivation and discipline to fiqh lesson. International Journal of Scientific & Technology Research, 2(2), 108–112. Lee, J., Kao, H.-A., & Yang, S. (2014). Service innovation and smart analytics for industry 4.0 and big data environment. Procedia CIRP, 16, 3–8. https://doi.org/10.1016/j.procir.2014.02.001. IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 208 Manurung, Paisal, Aryni, Y. (2019). Profile evaluation in Indonesia: The encouragement of educational change facing the era of digitalism. Journal of Educational Research and Evaluation, 3(4), 217-223. Marolt, M., Pucihar, A., & Zimmermann, D. H. (2015). Social CRM adoption and its impact on performance outcomes: A literature review. Organizacija, 48(4), 260–271, http://doi.org/10.1515/orga-2015-0022. Morrar, R., Arman, H. & Mousa, S. (2017). The fourth industrial revolution (Industry 4.0): A social innovation perspective. Technology Innovation Management Review, 7(11), 1-20. National Academy of Sciences. (2011). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. Washington DC: The National Academic Press. Richard, H. (2018). Managing problem solving learning for developing students’ creativity in classroom. Journal of Education for Teaching, 3(14), 113-121. Roberts, A. (2012). A justification for STEM education. Technology and Engineering Teacher, 74(8), 1-5. Sagala, R., Rofiqul, U., Thahir, A., Saregar, A. Wardani, I. (2019). The effectiveness of STEM-based on gender differences: The impact of physics concept understanding. European Journal of Educational Research, 8(3), 753-761. Sudiarta, I. G. P., & Widana, I. W. (2019). Increasing mathematical proficiency and students character: lesson from the implementation of blended learning in junior high school in Bali. IOP Conf. Series: Journal of Physics: Conf. Series1317 (2019) 012118, doi:10.1088/1742-6596/1317/1/012118. Suarimbawa, K. A., Marhaeni, A. A. I. N. , Suprianti, G. A. P. (2017). An analysis of authentic assessment implementation based on curriculum 2013 in SMP Negeri 4 Singaraja. Journal of Education Research and Evaluation, 1(1), 38-45. Stout, J. G., Dasgupta, N., Hunsinger, M., & McManus, M. A. (2011). STEM-ing the tide: Using ingroup experts to inoculate women’s self-concept in science, technology, engineering, and mathematics (STEM). Journal of Personality and Social Psychology, 100(2), 255-270. Sun, J. C. Y. & Wu, Y. T. (2016). Analysis of learning achievement and teacher-student interactions in flipped and conventional classrooms. International Review of Research in Open and Distance Learning, 17(1), 79–99, https://doi.org/10.19173/irrodl.v17i1.2116. Wang, MT., & Degol, J.L. (2017). Gender gap in science, technology, engineering, and mathematics (STEM): Current knowledge, implications for practice, policy, and future directions. Educ Psychol Rev 29, 119–140, https://doi.org/10.1007/s10648-015-9355-x. Weinstein, S & Preiss, D. (2017). Scaffolding to promote critical thinking and learner autonomy among pre-service education students. Journal of Education and Training, 4(1), 69-87. doi:10.5296/jet.v4i1.9871. Widana, I. W., Suarta, I. M., Citrawan, I. W. (2019). Application of simpang tegar method: Using data comparison. Jour of Adv Research in Dynamical & Control Systems, 11(2), 1825-1832. Widana, I. W. (2020). The effect of digital literacy on the ability of teachers to develop HOTS-based assessment. Journal of Physics: Conference Series 1503(2020), 012045, doi:10.1088/1742-6596/1503/1/012045. IRJE |Indonesian Research Journal in Education| |Vol. 5| No. 1|June|Year 2021| |E-ISSN: 2580-5711|https://online-journal.unja.ac.id/index.php/irje/index| 209 Biographical notes I WAYAN WIDANA is a lecturer at the Mathematics Education Study Program, Faculty of Teacher Training and Education, PGRI Mahadewa University, Indonesia, Denpasar, Bali, Indonesia; i.wayan.widana.bali@gmail.com AGUS TATANG SOPANDI is a lecturer in Elementary School Teacher Education Study Program, Teacher Training and Education Faculty, Universitas Terbuka; UPBJJ Denpasar, Bali; atatang@ecampus.ut.ac.id GEDE SUWARDIKA is a lecturer at the Statistics Study Program, Faculty of Science and Technology, Universitas Terbuka; UPBJJ Denpasar, Bali; isuwardika@ecampus.ut.ac.id mailto:i.wayan.widana.bali@gmail.com mailto:atatang@ecampus.ut.ac.id mailto:isuwardika@ecampus.ut.ac.id