Microsoft Word - 171-177 Computer | 171 Ibn Al-Haitham Jour. for Pure & Appl. Sci. IHJPAS https://doi.org/10.30526/31.3.2019 Vol. 31 (3) 2018 The Computer Based Tests: A Digital Substitution for the Iraqi Postgraduate Students Ammar Awni Abbas Baghdadi Computer Center, University of Baghdad, Baghdad, Iraq Ammarawni@uob.edu.iq Article history: Received 1October 2018, Accepted 17 October 2018, Published December 2018 Abstract The wide spread of Computer Based Test in the Iraqi universities raised an important concern for some scholars thinking about integrating computers as an academic assessment tool before making big change in the examination systems, that concern is: how the computerized test score is effected in comparison to old paper and pencil. This paper researches whether test scores of Iraqi students varies in the computer, Pencil based tests, sixty-four postgraduate graduates from Iraqi universities. Thirty-five female and twenty- nine males, all graduated from colleges of science in specializations other than computer science and applying for postgraduate studies in the Iraqi public universities. Therefore, are obliged to get the Iraqi computer skills certification for the postgraduate studies, those students chose the computer center in the University of Baghdad to get that certificate. The findings of the study confirmed that postgraduate candidates test scores were not distinctive in the PC based test and in the paper based test, which drove us to presume that PC based testing can be considered as a promising elective method for the logical postgraduate candidates in Iraq. Keywords: PC Based Tests, CBT, Paper Based test, PPT. 1. Introduction The strategy for managing tests in which the reactions are electronically recorded, evaluated, or both is called Computer-Based Testing (CBT), Computer directed testing, Computer-Based Assessment or e-exam. CBT might be an independent framework or a piece of a virtual learning system, potentially got through the World Wide Web. For financial, practical advantages with immediate grading; the Ministry Of Higher Education and Scientific Research in Iraq (MOHESR) progressively moving their Pencil-Paper tests (PPT) to PC based tests CBT. The reaction of the students is blended and plainly connects with singular contrasts related with PC encounter, PC nervousness and PC dispositions [1]. Prior computerized papers were focused on looking at people's scores on the Internet-based analysis to studies finished with paper-and-pencil [2, 3]. The following trend of papers considered the key factors between the two types such as content familiarity, computer familiarity, competitiveness and gender [4, 5]. Reviews the purposes behind utilizing PCs as an evaluation tool in some tests. Results demonstrate that arbitrary based tests can have various significant leverage over settled evaluations; likewise, CBT does not need to be seen in disconnection from the learning condition, yet, can have an effect upon an understudy's examination methodology. CBT can be used in all disciplines of science [6] surveyed under postgraduate medical education students about their preference of CBT and PPT and the majority of students preferred CBT. Computer | 172 Ibn Al-Haitham Jour. for Pure & Appl. Sci. IHJPAS https://doi.org/10.30526/31.3.2019 Vol. 31 (3) 2018 The speed and performance differences where investigated by [7]. The result showed that Students finished the PC based appraisal is faster than the paper-based evaluation, with no distinction in scores and different computer interfaces doesn't affect the score of the students. The testing effects, scores, time and motivations were investigated by [8]. This study has employed a Solomon four-group experimental design. PPT was a significant impact on time and motivation of the test by the testing effects. CBT was more dependable as far as interior and outer validities and it decreased testing time and expanded testing desire of the students. Some studies focused on issues considering equivalence issues between the media by reviewing performance measures [9]. Another form of utilizing computer assessment is "Digital Ink" as suggested by [10]. Students might be hindered by the presentation of online appraisal, except if mind is taken with the student– evaluation interface as [11] found. Two researches are significantly important and closely related to our work. Firstly, [12] demonstrated that test scores of undergrad Turkish understudies were not distinctive in CBT and PPT. Secondly, [13] compares the test results in CBT and PPT in Iranian English language learners. The conclusion of the study demonstrates the superiority of PPT over CBT. 2. Iraqi CBT Students applying for the postgraduate studies in Iraqi universities must show their proficiency in computer skills as part of the requirements for the admission. In 2009 this test was made solely by a pencil and paper. After that period, IC3 Digital Literacy Certification was introduced to accomplish that task. In 2018 the Admission examinations for the postgraduate student for 11 departments in 10 universities across Iraq were included in the Computer Based Tests, for both scientific and humanitarian colleges. Iraq Ministry of Higher Education and Scientific Research (MOHESR) with the aim to guarantee every postgraduate candidate has the basal computerized proficiency abilities to prevail in a worldwide economy have authorized in 2014. A Java Script Computer Based Tests (CBT) that was written in the computer center in the University of Baghdad and reviewed by Specialized Committee from MOHESR. Iraqi certificate named "Iraqi computer skills and information technology" were used in all Iraqi official Universities as an assessment tool for computer skills to replace the IC3 certificates. This study aims to tackle the issue of transforming from pencil and paper test (PPT) to CBT for the Iraqi students. The coming questions were developed for the end goal of examination: 1. Is there a variation in the test scores of the elected Iraqi postgraduate candidates in CBT and PPT? 2. Is there a variation in the test scores of the elected male Iraqi postgraduate candidates in CBT and PPT? 3. Is there a variation in the test scores of the elected female Iraqi postgraduate candidates in CBT and PPT? In general, the types of questions in both exams is very similar to each other, for example both exams have multiple choice questions (MCQ), TRUE and FALSE and practical exams. Computer | 173 Ibn Al-Haitham Jour. for Pure & Appl. Sci. IHJPAS https://doi.org/10.30526/31.3.2019 Vol. 31 (3) 2018 3. Method 3.1. Instructional setting This study was conducted at the University of Baghdad with postgraduate applicant students applying for postgraduate studies in pure science (excluding computer science) such as biology, physics, chemistry, geology, mathematics and astronomy. Postgraduate candidates had finished the instructive course taken in the computer center before the study was conducted. This examination concentrated on deciding if the test scores of Iraqi postgraduate candidates are distinctive in the Iraqi CBT and PPT. Results are used to answer the question in the investigation. 3.2. Participants The participants were postgraduate candidates applying for postgraduate position in Iraqi public college (since the exam is national and any university test center can be accepted as a source of the Iraq computer skills certificate). The student was selected to the postgraduate studies according to their scores in the undergraduate studies (average applicant students). Students applied to the University of Baghdad from the diverse parts of Iraq. Information were gathered in early 2018 and included sixty-four college of science student (excluding computer science). 3.3. Instrument Each student was subjected to two exams, first the Iraqi CBT for computer skills to investigate students’ knowledge in computers in general. Question types ranges from multiple-choice, TRUE or FALSE and practical small tests, after one hour break the students were asked to take a second PPT. All students attended three weeks training course taught by specialized instructors (only one course is needed since the study material conformed for the two exams). The study materials include: 1-Computing Fundamentals.2-KeyApplications.3- Living Online. In the Iraqi certificate of computer skills and information technology, all these parts were merged together. The total of a hundred degrees is the total degree earned by the typical answer distributed evenly with no penalty on wrong answers. The Iraqi CBT permit instant scoring of the test at the end, while the PPTs where graded by special instructors. To make students acquainted with the Iraqi CBT condition, an example CBT exercise test was additionally created. In the wake of finishing the CBT, students could see their scores on PCs. Partaking students' scores in the CBT were recorded by the examination colleague. Members of the study were given one hour for each type of the test. 3.4. Analysis The results of the CBT and PPT versions of the test were uploaded into a Microsoft Excel 2013 for later analysis. ANOVA and t-test was utilized to test the three theories. All statistical analysis reported in this paper were directed with a significant level of (0.05). 4. Results Data was collected from twenty-nine males and thirty-five female postgraduate candidates. The dispersion of results in the PPT and in the CBT forms of the test is displayed at the Table 1. Computer | 174 Ibn Al-Haitham Jour. for Pure & Appl. Sci. IHJPAS https://doi.org/10.30526/31.3.2019 Vol. 31 (3) 2018 Table 1. Statistical results of CBT and PPT. N Mean Std.Dev Std.Error 95% confidence interval for mean Min Max CBT 64 70.45 12.24 1.56 3.057 50 93 PPT 64 71.25 11.84 1.48 2.958 50 95 Total 128 70.85 12 1.06 2.09 50 95 The first research question, is there a variation in the test scores of the elected Iraqi postgraduate candidates in CBT and PPT. ANOVA was unable to dismiss the first null hypothesis that test scores of elected Iraqi postgraduate candidate’s students did not vary in the PPT and CBT (F = 0.14, p > 0.05). Students’ scores did not differ in the PPT and CBT version of the test (Table 2.). (Note: Df or Degree of freedom is the number of values in the final calculation of a statistic that are free to vary). The mean of the square is greater than or equal to the square of the mean, from Jensen's inequality. Table 2. One Way ANOVA between CBT and PPT. Sum of Squares Df Mean Square F Between Groups 20.32 1 20.32 0.14 Within Groups 18275.85 126 145.04 Total 18296.17 127 To answer second research question of the study, is there a variation in the test scores of the elected male Iraqi postgraduate candidates in CBT and PPT. The dissemination of male students' outcomes in the CBT and PPT variants of the test is exhibited in Table 3. Table 3. Statistical results of Male CBT and PPT. N Mean Std.Dev Std.Error 95% confidence interval for mean Min Max CBT 29 70.44 12.63 2.34 4.8 50 93 PPT 29 73 11.79 2.18 4.48 54 95 Total 58 71.72 12.18 1.59 3.2 50 95 Results of the ANOVA failed to dismiss the second null hypothesis that test scores of elected male postgraduate Iraqi students did not vary in the PPT and CBT (F = 0.63, p > 0.05). Male students’ scores were not statistically variant in the PPT and CBT, as exhibited in Table 4. Table 4. One Way ANOVA between Male CBT and PPT. Sum of Squares Df Mean Square F Between Groups 94.41 1 94.41 0.63 Within Groups 8363.17 56 149.34 Total 8457.58 56 Computer | 175 Ibn Al-Haitham Jour. for Pure & Appl. Sci. IHJPAS https://doi.org/10.30526/31.3.2019 Vol. 31 (3) 2018 The last research question, is there a variation in the test scores of the elected female Iraqi postgraduate candidates in CBT and PPT. The female participants’ scores in the PPT and CBT versions of the test is exhibited in Table 5. Table 5. Statistical results of Female CBT and PPT. N Mean Std.Dev Std.Error 95% confidence interval for mean Min Max CBT 35 70.45 12.09 2.04 4.15 50 88 PPT 35 69.8 11.85 2 4.07 50 91 Total 70 70.12 11.89 1.42 2.83 50 91 ANOVA failed to dismiss the null hypothesis that test scores of elected female Iraqi postgraduate students did not vary in the PPT and CBT test (F = 0.0527, p > 0.05). Female students’ scores did not vary statistically in the CBT and in the PPT versions of the test, as exhibited in Table 6. Table 6. One Way ANOVA between Female CBT and PPT. Sum of Squares Df Mean Square F Between Groups 7.55 1 7.55 0.0527 Within Groups 9750.2 68 143.3 Total 9757.8 69 An independent samples t-test is utilized to match the means of the normally distributed interval dependent variable for the two independent groups CBT and PPT. Since two means in the investigation originated from similar subjects, matched t-test formula was utilized to compare the mean scores of the students on both tests, as exhibited in Table 7, the t-observed value is 1.99 at P < 0.05. This amount of t-value at 63 (N-1) degrees of freedom in a<.05 is greater than the critical value of t, i.e. 1.98. The outcomes of inferential analysis demonstrated a significant difference between the students' mean scores on CBT and PPT. Table 7. Matched T-Test between CBT and PPT. T DF T. critical CBT vs. PPT 1.97 126 1.65 5. Discussions A tremendous effort is made to put the PPT exams together and a lot of work is made to elevate these exams to the Iraqi CBT standards. The fundamental reason for this investigation is to compare Iraqi postgraduate students’ performance in PPT and CBT. The results also seem to be consistent with the results of [12]; and contradict with the finding of [13], elected students’ scores in this study did not differ in the PPT and CBT versions of the test. For the question of whether the gender plays an important role in the performance of the student participants, no noteworthy distinction was found in the two types of exams. In a developing country like Iraq Computer | 176 Ibn Al-Haitham Jour. for Pure & Appl. Sci. IHJPAS https://doi.org/10.30526/31.3.2019 Vol. 31 (3) 2018 where the use of modern technology in education is made very recently. The need to make an early assessment for the benefits and drawbacks of the whole practice is very useful and help in preventing and anticipating future errors. The time is a very important factor to consider in CBT, but not considered in this study based on previous observations made by test centers supervisors. Students didn’t complain about the time to finish the exam with the review that if possible in both CBT and PPT [14]. Earlier computer experiences didn’t have an impact on the test score. Since all Iraqi students are considered to have some degree of experience from their first year of study in the undergraduate studies, computer skills are taught as a mandatory course. This study is made on special specimen of students where all applicants are applying for the postgraduate studies in pure science. Students are considered to be the average students in the scientific fields in Iraq, but the results cannot be generalized for all student. The most prominent type of question in the exam is the MCQ type with less True or False questions, other forms such as Fill in the blanks and reorder should be used. The finding of the study shows that we can use the CBT without any reservations or hesitations in scientific fields of study in Iraqi Universities. In spite of the most common disadvantages of CBT like: computer glitches, errors in content, and security lapses and security breaches, the advantage CBT overcomes its disadvantages [15]. Immediate results, choice of taking practice tests at whatever point and testing at home or anyplace they need are a portion of the favorable circumstances of CBT. The instructor, can design different versions of the exams and assignments without having to physically screen which understudies got which tests are rapidly assessing the execution of the test, all information can be put away on a solitary server, or a local computer [16]. The disadvantages include: the possibility of guessing, no ability to assess the steps of execution, the possibility of guessing in the MCQ being 25 %( for 4 choices) and 50% for True or False type of questions, loss of Information in cases of system failures. CBT isn't appropriate for article composing and investigation or intellectual reasoning testing. References 1. McDonald. The impact of individual differences on the equivalence of computer- based and paper-and-pencil educational assessments, Computers and Education. 2002, 39, 3, 299–312. 2. Potosky, D.; Bobko, P. Computer versus paper-and-pencil administration mode and response distortion in noncognitive selection tests. Journal of Applied Psychology. 1997, 82, 2, 293-299. 3. Mead, A. D.; Drasgow, F. Equivalence of Computerized, Paper, Pencil Cognitive Ability Tests: A Meta-Analysis. Psychological bulletin. 1993, 114, 3, 449-458. DOI: 10.1037/0033-2909.114.3.449. 4. Clariana, R.; Wallace, P.; Paper-based versus computer-based assessment: key factors associated with the test mode effect. 2002, 33 - 5. 5. Thelwall, M. Computer-based assessment: a versatile educational tool, Computers and Education.2000, 32. 6. Lim, E.; Ong, B.; Wilder-Smith, E.; Seet, R. Computer-based Versus Pen-and-paper Testing: Students. 2006, Perception. Computer | 177 Ibn Al-Haitham Jour. for Pure & Appl. Sci. IHJPAS https://doi.org/10.30526/31.3.2019 Vol. 31 (3) 2018 7. Bodmann, S.M.; Robinson, D.H. Speed and Performance Differences among Computer-Based and Paper-Pencil Tests. Journal of Educational Computing Research. 2004, 31,1, 51-60. 8. Piaw, C. Comparisons Between Computer-Based Testing and Paper-Pencil Testing: Testing Effect, Test Scores. Testing Time and Testing Motivation, In Proceedings of the Informatics Conference at: University of Malaya. 2011, 1–9. 9. Noyes, J; Garland, K. Computer- vs. paper-based tasks: Are they equivalent, Ergonomics. 2008, 51, 9, 1352–1375. 10. Siozos, P.; Palaigeorgiou, G.; Triantafyllakos, G.; Despotakis, T. Computer Based Testing Using Digital Ink. Participatory Design of a Tablet PC Based Assessment Application for Secondary Education. Computers and Education. 2009, 52, 811-819. 11. Ricketts, C.; Wilks, S.J. Improving Student Performance Through Computer-based Assessment, Assessment and Evaluation in Higher Education. 2002, l27, 5, 475-459. 12. Akdemir, O.; Oguz, A. Computer-based testing: an alternative for the assessment of Turkish undergraduate students. Computers and Education. 2008, 51, 1198-1204. 13. Hosseinia, M.; Jafre Zainol Abidinb, M.; Baghdarniac, M. Comparability of Test Results of Computer Based Tests (CBT) and Paper and Pencil Tests (PPT) among English Language Learners in Iran. Procedia - Social and Behavioral Sciences. 2014, 98, 659 – 667. 14. Parshal, C.; Spray, J.; Kalohan, j.; Dave y, T. Considerations in Computer-Based Testing, 1st ed., Springer, New York 2002, ISBN 0-387-98731-2. 15. Bartram, D.; Hambleton, R. Computer-Based Testing and the Internet, 1st ed., Wiley, England, 2006, ISBN 0-470-01721-X. 16. Kuzmina, I. Computer –Based Testing: Advantages and Disadvantages, Bulletin of the NTUU "KPI". 2010, 28, 45, 192-196.