IJOLAE | p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 2 (1) (2020) 1-9 1 Rasch Modeling: A Multiple Choice Chemistry Test Rasch Modeling: A Multiple Choice Chemistry Test Atiek Winarti1, Al Mubarak2 1,2Faculty of Teacher Training Education, Universitas Lambung Mangkurat, Indonesia DOI: 10.23917/ijolae.v2i1.8985 Received: October 6th, 2019. Revised: October 23rd, 2019. Accepted: October 25th, 2019. Available Online: October 26th, 2019. Published Regulary: January 1st, 2020. Abstract The study aimed to reveal the difficulty level of items and the suitability of items of Chemistry test with the Rasch model. In addition to detecting this item quality, the Rasch model shows the student's answer pattern as well, so that the assessment can imply the quality of the instrument as an assessment of chemical learn- ing. As many as 20 numbers of multiple-choice questions in chemical bonding material were analyzed by using WINSTEPS 3.73. The samples consisted of 200 senior high school students in Banjarmasin Indonesia. The results revealed that the average item measure was 0.00 with items (Measure Order = 4.64) which has the highest difficulty level. The Q10 was the item that has a level of conformity with the model, and outliers or misfit in Rasch were MNSQ=+0.97, ZSTD=-0.2, Pt Mean Corr=+0.58. In other words, assessment of learning with test techniques such as multiple choice based on Rasch model analysis was an effective way for teachers to review the progress of students in the learning process, guidelines for designing chemical learning strategies, and identifying students' understanding of chemical material. Keywords: rasch model, multiple choices, chemical bonding Corresponding Author: Atiek Winarti, Faculty of Teacher Training Education, Universitas Lambung Mangkurat, Indonesia e-mail: atiekwin_kimia@ulm.ac.id 1. Introduction Learning assessment is an important point that must be conducted by the teacher (Potgieter & Davidowitz, 2011). The as- sessment of learning provided contains a lot of essential information such as evaluating student learning progress, the extent of stu- dents' cognitive depth on learning that has been passed, and the accuracy of test instru- ments in measuring their mental models (Potgieter &. Davidowitz, 2011; Brannon et al, 2018). The learning process that is as- sessed by the instructor without using as- sessment will be difficult to know cognitive development, behavior, depth of understand- ing, and the impact of the designed teaching (Potgieter &. Davidowitz, 2011). During the learning process, the instruc- tor will not only transform knowledge in the classroom but also provide other treatments to develop students' potential such as as- signments, practice cognitive skills, forma- tive or summative tests, and direct communi- cation between the teacher and students sci- entifically (Sprague et. Al, 2018). Indirectly, that learning assessment is an illustration of how the teacher reflects on the learning pro- cess that has been experienced (Izci et al., 2018). Associated with the teaching of chem- istry, that the main target of learning is not only to bring students at a high cognitive level but also how students interpret and in- ternalize experiences that have been experi- enced while learning to become valuable individuals (Hindal, 2013). A written test as an assessment tech- nique is often used to review the progress of student learning and the effectiveness of learning undertaken (Herrmann-Abell & DeBoer, 2011). Written tests that can be used are multiple choice with the aim of analyzing Indonesian Journal on Learning and Advanced Education http://journals.ums.ac.id/index.php/ijolae Vol. 2 (1) (2020) 1-9 IJOLAE | p-ISSN 2655-920x, e-ISSN 2656-2804 2 Rasch Modeling: A Multiple Choice Chemistry Test the cognitive processes of students, empha- sizing whether students experience miscon- ceptions or not on chemical material, and evaluating learning concepts, so that assess- ment results are used as references in updat- ing the learning process, and identifying the nature of the items distributed (Brandriet & Bretz, 2014; Cheung, 2011; Treagust, et al, 2011; Milenković, et al, 2016; Yasin, et al, 2015). Experts also emphasized that the use of a distractor for multiple choice instru- ments could be used as an advantage in mak- ing multiple choice items (Herrmann-Abell & DeBoer, 2011; Villafañe, 2011). Multiple choice questions with distractors can diag- nose students' ability to understand material and this becomes a strategy in preventing potential student misconceptions (Herrmann- Abell & DeBoer, 2011). In addition, the in- structor also made the results of cognitive chemical analysis in this format as a guide in evaluating the learning process (Rauch & Hartig, 2010). Measurements with the multiple choice question format are closely related to the ability of the instrument to measure students' cognitive structure and item quality (de la Torre, 2009; Zamri, 2015). That is, the aspects of validity and reliability are part of the interpretation of data, especially the quality of the items (Zamri, 2015; Brandriet et.al, 2015). The Rasch model is a data analysis technique where it is very effective, precise, and systematic in justifying items with a logarithmic approach (Yasin et. al, 2015; Park et al. 2017; Lee et al., 2011). The concept of the Rasch model is not just a technique that shows the level of difficulty and feasibility of the question assessment instrument, but it is also able to show the pattern of student answers in responding to the problem (Sumintono, 2018; Chiang, 2015). In other words, the use of the Rasch model confirms that each student has the same opportunity to answer questions correctly, not just about items (Park et al. 2017; Chan et al, 2014). Rasch not only assesses the ability of students through the distribution of data, but also shows the level of difficulty of the problem, the suitability of the items with the sample used, and the symptoms of misconception, so that the Rasch model is an appropriate analysis technique used to identify the things mentioned (Zamri, 2015). The conclusion is that the Rasch model provides a more comprehensive and concrete picture in the measurement aspects of the test because the Rasch model involves two basic aspects as parameters namely (a) students' abilities, and (b) the difficulty level of the question or ability of the question (Zamri, 2015; Amin et al, 2012; Runnels, 2012). 2. Method The quantitative approach was used. It was used because the instructor needed to ascertain and confirm what and how the assessment process is carried out so that it required a quantitative and in-depth interpretation of the research conducted (Potgieter & Davidowitz, 2011). The selected samples are 200 students of chemistry education department from semester 1 to semester 5. These samples were considered important to identify their potential as prospective teachers. The assessment instrument of Chemical bonding material consisted of 20 number of multiple choice questions were analyzed by using WINSTEPS 3.73 Rasch model. The questions analyzed would represent the level of students' understanding of the material and also determined the measurement of the questions. Rasch modeling with multiple choice formats uses and combines an algorithm that states the results of probabilistic expectations of items "i" and respondents "n", which are mathematically expressed as (Chan, 2014; Runnels, 2012): Pni (Xni = 1ǀ bn, di) = Where Pni (Xni = 1ǀ bn, di) is the probability of respondent n in item i to produce a correct answer (x = 1); with the respondent's ability, βn and the difficulty level of the item δi. The equation above by Rasch can be further simplified by entering the logarithmic function and making it: IJOLAE | p-ISSN 2655-920x, e-ISSN 2656-2804 Vol. 2 (1) (2020) 1-9 3 Rasch Modeling: A Multiple Choice Chemistry Test Log Pni (Xni = 1ǀ βn, δi) = βn - δi So the probability will be a success that can be written as: Probability to succeed = respondent's ability - item difficulty level The Rasch model emphasizes that each student has the same opportunity to answer questions correctly and at the same time the problem has different levels of difficulty. This is termed Rasch as a person logit and logit item. Person logit = ln[p/(1-p)] Item logit = ln [p-value] = ln [p-value/(1-p- value)] 3. Result and Discussion a. Item Measure The item measure in Rasch is an analysis of the difficulty level of the item. The rightmost column (item) is the code of the 20 items distributed. This table provides a lot of information about the items distributed such as the "measure" column, where the column visualizes the difficulty level of each item with the term “logit value”. From top to bottom is the range from the difficulty level of the highest item to the lowest level. It means item with code Q15 is the item with the highest difficulty level (logit = 4.64) and Q7 is the item with the lowest difficulty level (-3.45 logit). Evidence that Q15 is the most difficult question, namely in the "total score" column, where Q15 has a total score of 7. This score indicates that overall only 7 samples answered questions Q15 correctly. Conversely, item Q7 has a total score of 192, meaning that 192 samples are able to answer Q7 questions correctly. In other words, Q7 is a question that is easily answered by students. Then, the table can also explain the specifications of the items in another perspective because Rasch produces the same scale distance. If you pay attention to the "logit value" of the question, they (questions) have different logit values between one another. For example, the Q19 (logit value = 1.93) when compared to the Q17 (logit value = .53) and the Q5 (logit value = 14), it can be said that the Q19 has a difficulty level 3 times compared to the Q17 and more than 10 times the Q5. Besides the distance of logit values differed significantly, the total score value also looks much different, where only 57 samples answered correctly for item Q19, 107 samples answered correctly for item Q17, and Q5 as many as 121 samples answered correctly. Overall, the item measure table has provided concrete information about the condition of each item so that this can be the evaluation for the teacher and the students. Teacher can use this data as a guideline and reference in making the appropriate learning assessment (written test assessment), knowing the characters of each question, analyzing the extent of students' understanding of the chemical bonding material, and how each item is assessed as a learning assessment. b. Item Fit The “misfit order” in Rasch is an analysis of the suitability of the item. If we previously discussed the difficulty level of the item, then "order misfit" represents the item's suitability level. Item fit (misfit order) explains whether an item is functioning normally or not in making measurements. If the detected items are not fit, then the question indicates that there is a misconception between students and the items they are working on (Herrmann-Abell & DeBoer, 2011). The indicated items are not fit, it needs more in-depth discussion. It means the teacher needs to further analyze the problem structure and student answer patterns so that they can find a point of the problem. Information based on this table is very valuable for teachers to be a reference for improving the quality of teaching. In addition, this information is also able to prevent and deal with attacks of misconceptions that will arise next. Vol. 2 (1) (2020) 1-9 IJOLAE | p-ISSN 2655-920x, e-ISSN 2656-2804 4 Rasch Modeling: A Multiple Choice Chemistry Test Table 1. Item measure Table 2. Item Fit/Misfit Order Boone et al (2014) explained that checking the suitability of the items can be identified based on the Mean Square Outfit (MNSQ) value with a range of values of 0.5