Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 8, No. 6, 8833-8847 2024 Publisher: Learning Gate DOI: 10.55214/25768484.v8i6.3880 © 2024 by the authors; licensee Learning Gate © 2024 by the authors; licensee Learning Gate * Correspondence: omonijoojo@yahoo.com Discriminatory indices of ‘introduction to psychology’ multiple choice examination questions Jonathan Adedayo, ODUKOYA1, Dare Ojo, OMONIJO2* 1Department of Psychology, College of Leadership Studies, Covenant University, Ota, Nigeria; adedayo.odukoya@covenantuniversity.edu.ng (J.A.O.) 2Department of Sociology, Faculty of Social and Management Sciences, Hallmark University, Ijebu-Itele, Nigeria; omonijoojo@yahoo.com (D.O.O.) Abstract: This study investigated the discriminatory indices of the multiple choice questions of a compulsory undergraduate course (Introduction to Psychology) in a private Nigerian university. The main research question raised was: Did all the items in the ‘Introduction to Psychology’ examination discriminate adequately between low scoring and high scoring students? To answer this question, the discriminatory index was derived for all the 70 items fielded in the examination. Though 255 students took this course, only students whose total scores fell within the topmost and lowest quartiles participated in this study. Students with missing data were extracted from the topmost and lowest quartiles. Consequently, the data of 100 students (50 in the topmost quartile and 50 in the lowest quartile) were utilized in computing the Discriminatory indices (Di) of the 70 Multiple Choice Questions. Out of the 70 items, two (2.9%) furnished poor Di, fifteen items (21.4%) had weak Di, fourteen items (20%) had fair Di, and thirty-nine items (55.7%) had fairly strong and strong Di. The findings are discussed and relevant recommendations made. Keywords: Discriminatory Index, Item Analysis, Multiple Choice Questions, Test Validity, Testing. 1. Introduction With the advent of pestilences like COVID-19 which authors such as [1] consider non-violent conflict that harms people's health and human development [2], the use of Multiple Choice Questions (MCQ), especially for online testing has increased. It is a common knowledge that essay examination questions are much easier to set than MCQs, while the case is reversed in scoring. It is often more laborious to score essay questions than MCQs. With a large student population, the wiser assessment format option should be MCQ. Diminishing returns tend to negatively affect the accurate scoring of a large pool of essay questions. Conversely, it is far easier and faster scoring MCQ for a large number of candidates with optical mark readers [OMR]. Another strong advantage of MCQ is that, if well- constructed with the application of Test Blueprint, it allows for good coverage of the subject curriculum cum syllabus. This way, the probability of achieving the curriculum objectives, and by extension the national educational objectives, is greatly enhanced [3-15]. The process of developing a valid MCQ often takes days, weeks and at times months, depending on the skill and experience of the test developers. It is for this reason that the strategy of item banking is often used in conjunction with the development exercise. The time and resources expended in developing good MCQs would hardly be justified when only a few items (questions) are written. It is for this reason that professional examination bodies like the Education Testing Service (ETS), West African Examinations Council, and Joint Admission and Matriculation Board, among others, ensure that a large pool of MCQs is developed, validated, and banked to last for about three years or more, depending on the life span of the curriculum. Another strong reason why it is imperative to empirically establish the validity of MCQs is because of the sensitivity of the decisions often made based on their results. Some are used for 8834 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 8, No. 6: 8833-8847, 2024 DOI: 10.55214/25768484.v8i6.3880 © 2024 by the authors; licensee Learning Gate recruitment, selection, admission, and promotion [16]. Invalid examination questions naturally invalidate the results and decisions made. Several people's destinies, hopes and expectations have been frustrated because of such invalid examination questions. Some even became depressed and committed suicide [17]. The core objective of this study therefore is to demonstrate one of the standard procedures for validating MCQs, using Discriminatory Index. Consequently, this study hopes to evolve recommendations that will further enhance Learning Facilitators’ skills for more effective implementation of item analysis, hence enhancing multiple choice objective test validity. Item analysis includes the Discriminatory Index. Beyond the Discriminatory Index, other standard procedures for ascertaining the validity of MCQ are done via computation of Difficulty Index and Distractive Index [17-19]. Foundational to item analysis procedures are content validation and strict adherence to Item writing rules. The procedure for establishing Difficulty and Distractive Indices has been treated by [20]. The study focuses on the Discriminatory Index. 1.1. Discriminatory Index The Discriminatory Index (Di) is the difference between the proportion of candidates who got an item right in the Upper Quartile (UQ) and those who got the answer to the same item right in the Lower Quartile (LQ) [21]. The discrimination index shows the degree to which an item discriminates between high and low-scoring students [21-27]. Di = 𝑈𝑄 𝑅𝑖𝑔ℎ𝑡 𝑈𝑄 𝑛 − 𝐿𝑄 𝑅𝑖𝑔ℎ𝑡 𝐿𝑄 𝑛 - Formula 1 Here is another formula from the [27]: The range of values for Di, therefore, is between 0 and ±1. The closer to zero and the more negative the value of Di, the poorer the Discriminatory Index. This implies that the item is not appropriately discriminating or differentiating high scoring and low scoring students. For instance, the literal meaning of a negative Di is that more of the students who scored higher on the overall (which suggests intellectually superior students) missed the answer to an item than the intellectually weaker students. This is and calls for a closer review of the item. Such review often reveals anomalies in the item that will likely require item moderation. In some cases, the item may be unredeemable and might need to be discarded outright. This is why Discriminant Index is more reliable empirical evidence of the content validity of multiple-choice examination questions [14]. Imagine that a large number of items in your MCQs have near zero or negative Di. Further imagine that the result of such an examination, being a compulsory course, might prevent affected students from getting promoted or graduating. The implications are better imagined than experienced. Depending on the personality and resilience of affected students, some may slip into depression and suicide ideation [17]. Consequently, the ideal psychometric requirement is to conduct item analysis and item moderation before final test administration. This is imperative for all MCQ examinations, be it elective or compulsory courses [18]. 8835 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 8, No. 6: 8833-8847, 2024 DOI: 10.55214/25768484.v8i6.3880 © 2024 by the authors; licensee Learning Gate 2. Method This study adopted the survey design. Live multiple choice questions (MCQ) on Introduction to Psychology were administered on undergraduate students in a private Nigerian University and the responses constituted the data for this study. Though two hundred and fifty-one students registered for the course, one hundred students were purposively sampled from the lower and upper quartiles to derive the Discriminatory Index. The mean age of the sample is 16 years. The participants were 100 level students in a private university in Nigeria. The main instrument used in this study was the 70-item multiple choice end-of-semester examination questions on Introduction to Psychology, a compulsory two-unit course. The course was examined solely with Multiple Choice Question. This examination constituted 70% while the Continuous Assessment constituted 30% of the whole course assessment. Extra care was therefore taken to ensure the Content Validity of this final examination. First, the relevance of all questions to the content and objectives of the course compact (called Course Outline or Scheme in other institutions) was checked. Thereafter, the stem (main question), keys (correct answer) and distractors (incorrect answers) in each item were also checked for adherence to the rules of item writing. Of importance is checking to ensure that the items were free of ambiguity, that there is only one absolutely correct answer for single- answer MCQ format, and that all the distractors were attractive enough to distract students that are prone to guessing. It is important to also mention that the Introduction to Psychology course was facilitated by three Lecturers (A Professor, Senior Lecturer and an Assistant Lecturer). The Senior Lecturer is a Test and Measurement major. Under strict examination conditions, the questions were administered within a duration of one hour. The students responded on Optical Mark Reader (OMR) forms which was later scanned and scored digitally. The output of the scanned OMR constituted the main data for this study. The output indicated the options selected by each student. With this information, it was possible to derive the Distractive Index (Di) for all the 70 items in the examination, using Formula 1 above. In conducting data analysis, Excel was used to arrange the total scores for all registered students (251) in descending order. Thereafter, the distribution was divided into four quartiles. The exercise made it possible to identify the students falling into the Upper and Lower Quartiles. With the elimination of cases having missing data, the lower and upper quartiles were brought down to 50 cases each, making it 100 cases altogether. The data was thereafter transferred into SPSS for derivation of frequency of respondents who got each item right in the Upper Quartile (UQ) and Lower Quartiles (LQ). Thereafter, the LQ proportion was deducted from the UQ proportion to determine the Discriminatory Index. The decision rules applied on the range of indices derived are the following: All items with negative Di were described as ‘Poor’, while items with Di ranging from 0.00 and 0.19 were described as Weak. Items with Di in the range of 2.0 and 3.9 were classified as Fair, while items with Di in the range of 4.0 and 5.9 were classified as Fairly Strong. Items with Di in the range of 6.0 and 7.9 were classified as Strong, while items with Di ranging from 8.0 and 1.0 were classified as Very Strong [23]. The summary of this classification is in Table 1 while the results of the Di derivations are presented Table 2a and 2b below. Table 1: Interpretation of discriminatory indices range. Di Range Interpretation -ve indices Poor 0.0-0.19 Weak 0.20-0.39 Fair 0.40-0.59 Fairly strong 0.60-0.79 Strong 0.80-1.0 Very Strong 8836 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 8, No. 6: 8833-8847, 2024 DOI: 10.55214/25768484.v8i6.3880 © 2024 by the authors; licensee Learning Gate 3. Results Table 2a: Discriminatory Indices of first 35 items in Introduction to Psychology Course. Upper Quartile [UQ] Lower Quartile [LQ] Di Item No. No. Correct UQ [n] UQ Prop No. Correct LQ [n] LQ Prop UQ-LQ Interpretation 1 37 50 0.74 36 50 0.72 0.02 Weak 2 41 50 0.82 34 50 0.68 0.14 Weak 3 43 50 0.86 11 50 0.22 0.64 Strong 4 43 50 0.86 24 50 0.48 0.38 Fair 5 47 50 0.94 17 50 0.34 0.60 Strong 6 45 50 0.90 7 50 0.14 0.76 Strong 7 46 50 0.92 25 50 0.5 0.42 Fairly Strong 8 43 50 0.86 9 50 0.18 0.68 Strong 9 46 50 0.92 19 50 0.38 0.54 Fairly Strong 10 27 50 0.54 7 50 0.14 0.40 Fairly Strong 11 41 50 0.82 14 50 0.28 0.54 Fairly Strong 12 24 50 0.48 23 50 0.46 0.02 Weak 13 46 50 0.92 9 50 0.18 0.74 Strong 14 50 50 1.00 13 50 0.26 0.74 Strong 15 8 50 0.16 9 50 0.18 -0.02 Poor 16 21 50 0.42 1 50 0.02 0.40 Fairly Strong 17 19 50 0.38 10 50 0.2 0.18 Weak 18 4 50 0.08 8 50 0.16 -0.08 Poor 19 44 50 0.88 7 50 0.14 0.74 Strong 20 48 50 0.96 22 50 0.44 0.52 Fairly Strong 21 45 50 0.90 16 50 0.32 0.58 Fairly Strong 22 37 50 0.74 22 50 0.44 0.30 Fair 23 12 50 0.24 6 50 0.12 0.12 Weak 24 24 50 0.48 21 50 0.42 0.06 Weak 25 49 50 0.98 33 50 0.66 0.32 Fair 26 47 50 0.94 37 50 0.74 0.20 Weak 27 43 50 0.86 32 50 0.64 0.22 Weak 28 42 50 0.84 11 50 0.22 0.62 Strong 29 42 50 0.84 10 50 0.2 0.64 Strong 30 41 50 0.82 32 50 0.64 0.18 Weak 31 48 50 0.96 33 50 0.66 0.30 Fair 32 36 50 0.72 12 50 0.24 0.48 Fairly Strong 33 44 50 0.88 19 50 0.38 0.50 Fairly Strong 34 49 50 0.98 23 50 0.46 0.52 Fairly Strong 8837 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 8, No. 6: 8833-8847, 2024 DOI: 10.55214/25768484.v8i6.3880 © 2024 by the authors; licensee Learning Gate 35 43 50 0.86 17 50 0.34 0.52 Fairly Strong Table 2b: Discriminatory Indices of last 35 items in Introduction to Psychology Course. ` Upper Quartile [UQ] Lower Quartile [LQ] Di Item No. No. correct UQ [n] UQ Prop No. Correct LQ [n] LQ prop UQ-LQ Interpretation 36 8 50 0.16 2 50 0.04 0.12 Weak 37 44 50 0.88 21 50 0.42 0.46 Fairly Strong 38 41 50 0.82 14 50 0.28 0.54 Fairly Strong 39 41 50 0.82 17 50 0.34 0.48 Fairly Strong 40 36 50 0.72 20 50 0.4 0.32 Fair 41 36 50 0.72 20 50 0.4 0.32 Fair 42 34 50 0.68 13 50 0.26 0.42 Fairly Strong 43 46 50 0.92 15 50 0.3 0.62 Strong 44 49 50 0.98 16 50 0.32 0.66 Strong 45 41 50 0.82 13 50 0.26 0.56 Fairly Strong 46 45 50 0.90 14 50 0.28 0.62 Strong 47 6 50 0.12 1 50 0.02 0.10 Weak 48 44 50 0.88 9 50 0.18 0.70 Strong 49 23 50 0.46 4 50 0.08 0.38 Fair 50 39 50 0.78 8 50 0.16 0.62 Strong 51 25 50 0.50 10 50 0.2 0.30 Fair 52 9 50 0.18 9 50 0.18 0.00 Weak 53 34 50 0.68 27 50 0.54 0.14 Weak 54 9 50 0.18 7 50 0.14 0.04 Weak 55 45 50 0.90 6 50 0.12 0.78 Strong 56 31 50 0.62 16 50 0.32 0.30 Fair 57 43 50 0.86 20 50 0.4 0.46 Fairly Strong 58 40 50 0.80 22 50 0.44 0.36 Fair 59 47 50 0.94 26 50 0.52 0.42 Fairly Strong 60 49 50 0.98 29 50 0.58 0.40 Fairly Strong 61 39 50 0.78 38 50 0.76 0.02 Weak 62 50 50 1.00 25 50 0.5 0.50 Fairly Strong 63 37 50 0.74 15 50 0.3 0.44 Fairly Strong 64 20 50 0.40 1 50 0.02 0.38 Fair 65 48 50 0.96 20 50 0.4 0.56 Fairly Strong 66 46 50 0.92 21 50 0.42 0.50 Fairly Strong 67 24 50 0.48 5 50 0.1 0.38 Fair 68 27 50 0.54 14 50 0.28 0.26 Fair 69 50 50 1.00 24 50 0.48 0.52 Fairly Strong 8838 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 8, No. 6: 8833-8847, 2024 DOI: 10.55214/25768484.v8i6.3880 © 2024 by the authors; licensee Learning Gate 70 29 50 0.58 14 50 0.28 0.30 Fair Table 3: Frequency & Percentage of Di Interpretations. Frequency Percent Poor 2 2.9 Weak 15 21.4 Fair 14 20.0 Fairly Strong 24 34.3 Strong 15 21.4 Total 70 100.0 From Table 3, it can be deduced that out of the 70 items, 2 (2.9%) exhibited poor Discriminatory Indices, with more Students in the Lower Quartile getting an item correct than the students in the Upper Quartile. The questions affected are items 15 [-0.02] and 18 [-0.08]. In all, twenty-nine items were weak and fair (29, 41.4%) while fourteen were fair (14, 20%). In all thirty-nine items were fairly strong and strong (39, 55.7%). It is important to note that there was no ‘very strong item’ in the whole lot, in terms of Discriminatory power. 4. Findings The following are the findings from this study: 1. Two items (2.9%) were deemed poor because they furnished negative Discriminatory indices. 2. Fifteen items (21.4%) were deemed weak. 3. Fourteen items (20%) were deemed fair. 4. Twenty-four items (34.3%) were deemed fairly strong. 5. Fifteen items (21.4%) were deemed strong. 5. Discussion 5.1. Two Items Were Deemed Poor Let’s have a closer look at the two questions that furnished negative Discriminant Indices - Items 15 and 18: 8839 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 8, No. 6: 8833-8847, 2024 DOI: 10.55214/25768484.v8i6.3880 © 2024 by the authors; licensee Learning Gate In item 15, the name of ‘Theodore Simon’ was misspelled as ‘Theodore Simon’. This could have accounted for the abnormal discriminatory index observed. The implication of this is that there is no correct answer in the options, hence the students resorted to guessing. This must have accounted for the sporadic responses. There is nothing wrong with item 18, apart from the missing question mark at the end of the question. The focus of the question is to test for the knowledge of the correct spelling of the neurotransmitter called ‘Acetylcholine’. It is also apparent many of the respondents are not familiar with the names of the other neurotransmitters listed in the options. The neurotransmitters were only mentioned during the lecture. They were not listed in the PowerPoint slides given to students. Consequently, many of the respondents, including the high-scoring students, did not know the correct spelling of ‘Acetylcholine’. They therefore resorted to guessing. This finding further suggests that many of the students, including the high-scoring ones, did not engage in deep study of the content of these courses. The discovery from item 18 therefore suggests that an abnormal Discriminatory index may not always be attributable to faulty multiple choice question. It could also be attributable to poor students’ preparation for examinations. 5.2. Fifteen Items Were Deemed Weak Items 1, 2, 12,17, 23, 24, 26, 27, 30, 36, 47, 52, 53, 54 and 61 were deemed weak in terms of Discriminatory power. Apart from item 47 that has a slight grammatical error, close scrutiny shows that all the other items listed here have sufficient content validity attributes. The reason adduced for the abnormal Discriminatory index obtained for item 18 above tend to apply for the items listed as weak here. It is not unlikely majority of the students, including those in the high scoring quartile, did not engage adequate study to correctly identify the correct answer. It is important to mention that the Lecturers, being conscious of the level of the students, hardly ask questions beyond the explanations made during lectures and the points in the PowerPoint slides they were given. 5.3. Fourteen Items Were Deemed Fair; Twenty-Four Items Were Deemed Fairly Strong; Fifteen Items Were Deemed Strong The fact that 75.7% of the questions were deemed fair, fairly, strong and strong is an indication that the examination under review could be graded A in terms of psychometric quality. However, there is clearly room for improvement. 5.4. General Observation The fact that there were still errors in some of the items in this examination, despite the partial psychometric intervention, reiterates the need to astutely apply item analysis before the live administration of all sensitive examinations of this nature. It is simply humanly impossible to spot all the errors in MCQ items solely via content validation visual review. It is important that all newly written MCQs, after thorough content validation, be taken through pilot testing with students of similar parameters, to generate data for item analysis. The empirical results obtained from the item analysis become reliable guide for item moderation. Subsequently, the approved items are banked for future use. This is the standard procedure for the use of Multiple Choice Objective questions. Any attempt at introducing short cut methods is bound to result in assessment errors which could prove to be debilitating to the recipients of such examination, and non-fulfillment of organisational and national developmental goals. 6. Conclusion This study investigated the discriminant validity of ‘Introduction to Psychology’ multiple choice examination questions in a Private Nigerian University. Though the number of faulty items were minimal, perhaps due to the intervention of a Psychometrician who took time to ensure adherence with the rules of item writing, yet the overall result shows that there is always need to conduct item analysis to prevent situations where some students will be put at undue disadvantage. Such students often know 8840 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 8, No. 6: 8833-8847, 2024 DOI: 10.55214/25768484.v8i6.3880 © 2024 by the authors; licensee Learning Gate the subject matter but may end up failing or get sub-standard result due to test development errors. In some cases, some students end up spending extra one year to make up for courses of this nature, which are often compulsory. Some students further develop depression as a result of such nasty experience. For these reasons, it is quite apparent that item analyses are a task that must be done with all sensitive multiple-choice objective examinations that are compulsory requirement for promotion, graduation and certification in all organisations and institutions handling such assessment exercises. Students should also be counselled and taught the procedure for deep study cum learning to prevent the kind of errors observed with item 18 in this study. Funding: We received the publication fees for this article from the Covenant University Centre for Research Innovation and Discovery (CUCRID). Copyright: © 2024 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). References [1] Omonijo, D. O., Obiorah, B. C., Uche, O. O. C., Anyaegbunam, M. C. Shaibu, A. O & Ogunwa, C. E. Exploring Social Theories in the Study of Insecurity in Contemporary Nigeria. The Journal of Social Sciences Research, 3(7): 59-76, 2017 [2] Adetola B. O & Omonijo, D. O. Attitude of Urban Dwellers to Waste Management and Control in Ogun State, Nigeria: A Developmental Challenge and Concern. International Journal of Scientific and Technology Research, 8(12): 874-881, 2019 [3] Kocdar, S., aarada N & Ahin M. D. Analysis of the Difficulty and Discrimination Indices of Multiple-Choice Questions According to Cognitive Levels in an Open and Distance Learning Context. The Turkish Online Journal of Educational Technology, 15 (4), 2016. [4] Anderson, L. W., & Krathwohl, D.R. (2001). A taxonomy for learning, teaching and assessing: a revision of bloom’s taxonomy of educational objectives. New York: Longman, 2001 [5] Biggs, J. B., & Collis, K. (1982). Evaluating the quality of learning: the SOLO taxonomy. New York, Academic Press, 1982 [6] Bloom, B. S. Taxonomy of educational objectives: the classification of educational goals. New York: Longmans, Gren and Company Inc., 1956 [7] Fink, L. D. Creating significant learning experiences: an integrated approach to designing college courses. San Francisco: Jossey-Bass, 2003 [8] Hannah, L. S., & Michaelis, J. U. A comprehensive framework for instructional objectives: a guide to systematic planning and evaluation. Reading, Mass: Addison- Wesley, 1977 [9] Marzano, R. J. Designing a new taxonomy of educational objectives. Thousand Oak, California: Corwin Press, Inc., 2001 [10] Stahl, R. J., & Murphy, G.T. The domain of cognition: an alternative Bloom’s cognitive domain within the framework of an information processing model. ERIC Documents Reproduction Service No: ED 208511, 1981 [11] Haladyna, T.M. Standardized achievement testing. Boston, MA: Allyn and Bacon, 2002 [12] Seaman, M. Bloom’s taxonomy: its evolution, revision, and use in the field of education. Curriculum and Teaching Dialogue, 13(1& 2), 29–43. 2011 [13] Krathwohl, D. R. A revision of Bloom’s taxonomy. Theory into Practice. 41(4), 212- 218, 2002 [14] Odukoya, J.A., Omonijo, D. O., Mistra, J & Ahuja, R. Advances in Intelligent Systems and ComputingVolume 1180 AISC, Pages 333 - 3422021 10th International Conference on Innovations in Bio-Inspired Computing and Applications, IBICA 2019 and 9th World Congress on Information and Communication Technologies, WICT 2019 Gunupur16 December 2019 through 18 December 2019 Code 24339, 2019 [15] Odukoya, J. A., Omonijo, D. O., Olowookere, E. I., John, M & Atayero, A.A. Admission policy in universities: In search of empirical evidence. International Journal of Scientific and Technology Research Volume 8, Issue 12, Pages 388 – 393, 2019 [16] Professional Testing Inc. Conduct the Item Analysis. Online, 2006 [17] Yusoff, M.S. Associations of pass-fail outcomes with psychological health of first- year medical students in a malaysian medical school. Sultan Qaboos Univ Med J. Vol.13(1):107-14. doi: 10.12816/0003203. Epub 2013 Feb 27. PMID: 23573390; PMCID: PMC3616775. 2013 [18] Sharif M.R., Asadi M.H., Sharif A.R., and Sayyah M. Examining the Multiple Choice Educational Examinations of College Students. Biomedical & Pharmacology Journal, 6 (1): 23-27, 2013 [19] Özçelik, D.A. Test hazrlama klavuzu. Ankara: ÖSYM Yaynlar, 1989 https://creativecommons.org/licenses/by/4.0/ 8841 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 8, No. 6: 8833-8847, 2024 DOI: 10.55214/25768484.v8i6.3880 © 2024 by the authors; licensee Learning Gate [20] Simonson, M., Smaldino, S., Albright, M., & Zvacek, S. Teaching and learning at a distance: foundations of distance education. 5th ed. Boston: Allyn & Bacon, 2012 [21] Zhang, W., Tsui, C., Jedege, O., Ng, F., & Kowk, L. A comparison of distance education in selected Asian open universities. Paper presented at 14th Annual Conference of Asian Association of Open Universities, Manila, Philippines, 2002 [22] Odukoya, J.A., Adekeye, O., Igbinoba, A.O., Afolabi, A. Item analysis of university- wide multiple choice objective examinations: the experience of a Nigerian private university. Quality and Quantity, 52(3): 983-997. DOI: 10.1007/s11135-017-0499- 2, 2018 [23] Maryani I, Prasetyo ZK, Wilujeng I, Purwanti S. Higher-order Thinking Test of Science for College Students Using Multidimensional Item Response Theory Analysis. Pegem Journal of Education and Instruction, 12(1): 292-300, 2022 [24] Linn, R.L. & Gronlund, N.E. Measurement and assessment in teaching (7th ed.). Englewood Cliffs, NJ: Prentice-Hall, 1995 [25] Hopkins, K.D., Stanley, J.C., & Hopkins, B.R. Educational and psychological measurement and evaluation (7th ed.). Englewood Cliffs, NJ: Prentice-Hall, 1990 [26] Burch, V. C.; Norman, G. R.; Schmidt, H. G.; van der Vleuten, C. P. M., (2008). Are Specialist Certification Examinations a Reliable Measure of Physician Competence? Advances in Health Sciences Education, 13(4): 521- 533. [27] University of Pretoria. Discriminatory Index. Pretoria: University of Pretoria - Online https://e.itg.be/bangalore/MCQ/Discrimination%20index.html, 2022 Appendix https://e.itg.be/bangalore/MCQ/Discrimination%20index.html 8842 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 8, No. 6: 8833-8847, 2024 DOI: 10.55214/25768484.v8i6.3880 © 2024 by the authors; licensee Learning Gate 8843 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 8, No. 6: 8833-8847, 2024 DOI: 10.55214/25768484.v8i6.3880 © 2024 by the authors; licensee Learning Gate 8844 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 8, No. 6: 8833-8847, 2024 DOI: 10.55214/25768484.v8i6.3880 © 2024 by the authors; licensee Learning Gate 8845 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 8, No. 6: 8833-8847, 2024 DOI: 10.55214/25768484.v8i6.3880 © 2024 by the authors; licensee Learning Gate 8846 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 8, No. 6: 8833-8847, 2024 DOI: 10.55214/25768484.v8i6.3880 © 2024 by the authors; licensee Learning Gate 8847 Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 8, No. 6: 8833-8847, 2024 DOI: 10.55214/25768484.v8i6.3880 © 2024 by the authors; licensee Learning Gate