46 JPAIR Multidisciplinary Research ABSTRACT Computer Aided Assessment (CAA) has been deployed in higher education to administer examinations in which the responses are electronically recorded, assessed, or both. CAA makes possible a detailed analysis of test results with minimal effort. It employed the descriptive and applied methods of research. A standardized research survey questionnaire was used. The study found out that 50.86% of the responses were computer aided, 43.43% were paper and pencil, 4.5% were board work and 1.4% was others. 45.14 of the responses were multiple choice, 22.29% were true or false, 17.14% were essay 14.86% were matching type and .57% were others. Test papers were physically distributed by the professors to students. HTML and CSS were used for GUI while PHP and MySQL for storing data. The Multiple Choice Test (MCT) software was evaluated usable. The study concludes that the students prefer computer-aided test as the delivery method of assessment. Moreover, students prefer multiple choice type of test especially when the assessment period is limited. The existing testing administration is manual. The combination of Hypertext Mark-up Language (HTML), Cascading Style Sheet (CSS), and Hypertext Pre-processor (PHP) and MySQL relational database management system is deemed effective in the development of a Multiple Choice Test Randomizer. Vol. 23 · January 2016 Print ISSN 2012-3981 • Online ISSN 2244-0445 doi: http://dx.doi.org/10.7719/jpair.v23i1.352 Journal Impact: H Index = 2 from Publish or Perish JPAIR Multidisciplinary Research is produced by PAIR, an ISO 9001:2008 QMS certified by AJA Registrars, Inc. Multiple Choice Test Randomizer of ICT Department of Abra State Institute of Science and Technology EXCEL PHILIP B. GUIDANG http://orcid.org/00000001-7080-8230 rieann12@gmail.com Abra State Institute of Science and Technology, Philippines 47 International Peer Reviewed Journal Keywords - Computer Aided Assessment, Multiple Choice Test, Randomizer, Online Assessment, descriptive-applied design, Abra, Philippines INTRODUCTION Computer Aided Assessment (CAA) has been practiced in higher education for the past five decades. Computer software was first created to test the behavior of student’s machine-language submissions. CAA is a method of administering examinations in which the responses are electronically recorded, assessed, or both. It offers an easy-to-use environment for preparing questions, conducting exams, and administering results. CAA makes possible a detailed analysis of test results with minimal effort. This can be used to identify areas within the course where the students have difficulty. As a result, alerting professor to the possible need to adapt their teaching. Or maybe it could be used to identify trends and patterns within the student group. Questions which are not successful at discriminating between students can readily be identified and improved for future years. Fielding (2003) states that CAA is the application of computers to assessment processes, including delivery of tests, and capture of responses. It is the use of computers to simplify evaluation (Constantine, 2008). These tests can be used for both summative and formative assessment. CAA may be stand alone and specific to machines within a computer laboratory based on a local network or web-based. The Georgia Department of Education (GaDOE) is providing the 2013-2014 release of the Georgia Online Assessment System (OAS). The OAS represents a dedicated resource for schools, districts, classroom teachers, students, and parents that allow for ongoing classroom instruction and student learning. The site allows educators to have access to test items aligned to the state mandated curriculum to develop assessments that inform teaching and learning. It allows students and parents transparency on the expectations placed on students about improving student achievement. Mitkov, Le An and Karamanis (2006) developed a computer-aided environment for generating multiple-choice test items. The software identifies ideas in the text and generates questions about ideas as well as multiple-choice distractors. Lately, with his desire to explicate generally the use of CAA to automate routine processes, Sangwin (2013) in his book entitled “Computer aided assessment of mathematics” explains how CAA is currently applied to assess mathematics. 48 JPAIR Multidisciplinary Research The Abra State Institute of Sciences and Technology (ASIST) is the only state college in the province of Abra. It is an agricultural school that offers Bachelor of Science in Agriculture and Bachelor of Science in Forestry. Moreover, it also offers science and technology courses such as Bachelor of Science in Mathematics (BSMath), Bachelor of Science in Biology (BSBIO), Bachelor of Science in Information Technology (BSIT). Also, it offers the courses Bachelor of Science in Secondary Education and Bachelor of Science in Elementary Education. Based on the records in the Office of the Registrar, there are 1900 students, as of June 2014 who were enrolled in all courses including the high school students. Faculty workload shows that a faculty member handles a maximum class size of fifty students. Assessment and evaluation are important parts of the teaching and learning process. Professors in the Abra State Institute of Sciences and Technology are required to conduct diverse forms of assessments. However, the administration of assessment is tiresome and cumbersome because the assessment is done manually. Based on observations, students often talk with their seatmates. This makes the professors keep on roaming around the room and try to warn the students and most of the time this would drain their energy. The researcher sought the opinions of three professors from the IT Department and has this to say, “In a lecture class having 50 students, most of the students tend to communicate to each other either by words or by a signal to send and receive answer(s). If a student caught several times doing it and seems to ignore warnings, he/she is asked to move out of the room and take another form of exam later.” After the examination proper, the hardship of the professors is not yet over because it is only the beginning. Individual checking and scoring of examinee’s test paper are even harder. It challenges the professor’s patience and perseverance to stay awake late at night to finish the task. Errors and inconsistencies in scoring are often encountered. Delays in checking and scoring due to hard to read penmanship prolong the distribution of examination result. In response to the above scenarios, the development of Multiple Choice Test Randomizer (MCTR) concept was instinctive, to provide a software solution that the professors and students need. MCTR is differed from the study of Mitkov, R., LE AN, H. A., & Karamanis, N. (2006) because professors who shall use MCTR will have to encode manually the questions by themselves and not MCTR. 49 International Peer Reviewed Journal OBJECTIVES OF THE STUDY The study developed the Multiple Choice Test Randomizer. Specifically, it aimed to: 1) Identify the preferred delivery method of assessment of the ICT Students; 2) Identify the preferred type of test of the ICT Students; 3) Identify the existing procedures of Testing Administration and Scoring; 4) Develop the Multiple Choice Test Randomizer (MCTR); 5) Determine the usability of Multiple Choice Test Randomizer in terms of usefulness, ease of use, ease of learning, and satisfaction. METHODOLOGY The researcher used both descriptive and applied methods of research. Descriptive method of research was used to narrate the delivery method of assessment, the preferred type of test of the ICT students, and the Existing procedures on testing administration and scoring. The study applied standardized test called USE Questionnaire to determine the usability of the software. Figure 1. The research paradigm Sources of data The respondents were two hundred fifty-one (251) ICT students and five (5) Faculty members. 50 JPAIR Multidisciplinary Research The freshmen and the sophomore students of both amalgamated (curriculum is synchronized with other state university and colleges in the Cordillera Administrative Region) and Ladderized of ICT Department were chosen as the respondents on the preferred type of delivery method of assessment and preferred type of test as they are still to be in the adjustment period in the college way of life while respondents to determine the usability of the developed Multiple Choice Test Randomizer. The third and fourth year ICT students were chosen because they have Software Engineering and Software Analysis and Design subjects wherein software development life cycles are discussed and they are also required to submit simple application programs and test its usability at the end of the semester. The ICT faculty members, too, were tapped to do a usability test of the software because they have the expertise on software development, user interface design, and because they are the direct users of the software. The Test Paper provides the information on components of Multiple Choice Test (MCT). The CSIT faculty members also provided the procedures for administration and scoring. The researcher personally floated and collected the questionnaire from the respondents to ensure a one hundred percent (100%) retrieval. The researcher asked permission from everyone who were surveyed. Table 1. Distribution of respondents Position Population Faculty 5 DIT 1a 33 DIT 1b 41 BSIT 1a (Amalgamated) 27 BSIT 1b (Amalgamated) 26 DIT 2a 27 DIT 2b 27 BSIT 2a 16 BSIT 2b 16 BSIT 1a (Ladderized) 25 BSIT 1b 22 BSIT 2a 22 BSIT 2b 22 Total 309 51 International Peer Reviewed Journal Data Instrumentation For objectives 1 and 2, the study used a survey questionnaire validated by 10 Agriculture and Education students. It was administered to identify the preferred delivery method of assessment and preferred type of test. For objective 3, the study used interview and observation. It was conducted to the ICT faculty members to determine the existing procedures for testing administration and scoring. The study also used documentary analysis on test paper having multiple choice test format. It was administered to determine the components of Multiple Choice Test (MCT). For objective 4, the software was created using the Incremental Build Model. The model consists of four stages: 1)Analysis; 2) Design; 3) Code, and 4)Test. Figure 2. Incremental Build Model Analysis Phase In this stage, the researcher identified the preferred delivery method of assessment and type of test of the ICT students. Likewise, identified the existing procedures of testing administration and scoring along with the components of a Multiple Choice Test (MCT). Design Phase In this phase, the researcher created a testing administration and scoring solution. User design and user experience were top priorities. Focused were on what the user’s need, what they value, their abilities, and also their limitations.  The goals and objectives of the IT Department were considered. On top of these considerations, the user design, and user experience best practices set the overall guidance. 52 JPAIR Multidisciplinary Research Coding Phase The coding stage consisted of the writing of computer readable instructions. The code was written using Hypertext Markup Language and Cascading Style Sheet to create Graphical User Interface or user presentation of the script. Hypertext Preprocessor (PHP) was used for server side scripting to collect data from the user and MySQL for storing user log or activity. Testing Phase Functional testing was undertaken. This was conducted by both teachers and students. It was conducted to check if the system works on the specified platform and with the required volume of data, and determines if the system meets the users’ requirements. For objective 5, a survey questionnaire was used from the concept of Lund (2001) known as USE questionnaire. USE stands for Usefulness, Satisfaction, and Ease of use. Subjective reactions to the usability of an application tend to be neglected for performance measures, and yet it is often the case that these metrics measure the aspects of the user experience that are most closely tied to user behavior and purchase decisions. Data Analysis Data from the respondents were interpreted using frequency count based on the highest number of responses to the preferred delivery method of assessment and type of test of ICT students. The usability of MCTR was tested with the application of USE questionnaire. Responses from the students and Faculty members were interpreted with the Likert Scale and Descriptive ratings: Point of Scale Mean Rating Descriptive Equivalent Rating Descriptive Interpretation 5 4.20-5.00 Strongly Agree Usable 4 3.40-4.19 Agree Usable 3 2.60-3.39 Neutral Usable 2 1.80-2.59 Disagree Not Usable 1 1.00-1.79 Strongly Disagree Not Usable The variables with responses within the mean range of 2.60 to 5.00 were interpreted as Usable while those variables with responses within the range 1.00 to 2.59 were interpreted as Not Usable. 53 International Peer Reviewed Journal RESULTS AND DISCUSSION Results showed that the 50.86% of responses from the ICT students were computer-aided, and only 43.4% were paper and pencil. This implies that the preferred delivery method of assessment of the students is computer-aided. It is for this reason that the researcher decided to create a computer aided assessment tool such as MCTR. This is in contrast with the study of Sheader, Gouldsborough and Grady (2006) where more students are “confident” about paper-based work as a method of assessment (91%) and only 50% made the same claim for CAA. On the hand option, “others” got the lowest responses. Two respondents indicated “actual or practical testing.” This means that the respondents preferred assessments of skills using application. In terms of preferred type of test, 45.14 % of the responses from the ICT students were Multiple Choice. This implies that students tend to like a type of test in which it requires a less of writing especially so when the assessment period is limited. It further implies that students like to perform an assessment which limits error on grammar and spelling. On the other hand, the option “others” got the lowest responses which is .57 % of the total responses. One respondent indicated enumeration. This means that the respondent is fond of memorizing lessons. The existing procedures of testing administration and scoring Based on personal interviews with faculty members, observation, and personal practice, the administration of testing begins when the professor formulates and encodes questions. The standard components of Multiple Choice Question are item number, question, and options based on the test paper analysis. Then, printing of the questionnaire followed. The professor may either reprint or photo copy in to reproduce the questionnaire. The testing administration proper is the actual distribution of test paper to the examinees. Afterward, the test papers are collected by the professor. Individual checking and scoring of test paper are next. It is in this part where professors spend a lot of time. It is also in this part wherein error and inconsistencies of scoring occur. The test result is distributed to the students a week after the exam. Description of Processes in the Multiple Choice Test Randomizer The software was developed by dividing the whole program into four separate modules. The four modules were: a) formulation and encoding of the question, 54 JPAIR Multidisciplinary Research b) registration and validation, c) user login and validation, d) testing and scoring and e) result. Figure 3. Encoding of Question In encoding questions, it is suggested that user must have to follow the format in Figure 3. The first column contains the question, followed by options form a to d. The next column fifth column contains the correct answer. Lastly, the sixth column contains the item number for each question. Figure 4. Saving the file in the CSV format After, the encoding of questions. Save the file in CSV format by clicking Save as type drop-down list, and then, select CSV (comma delimited). Type the file name and click Save button. 55 International Peer Reviewed Journal Figure 5. Uploading question to MCTR The Uploading of questions to MCTR begins by typing www.andwesign.org/ gui/upload.php in the Unified Resource Locator (URL). Select the drive where your file is located. Type the complete filename including the csv file extension. Click upload. Figure 6. Welcome page The welcome page is the first page that MCTR serves to all the computers that request for the service. It displays a text greeting along with the capabilities that it may offer for the user. Multiple users may be served at the same time. 56 JPAIR Multidisciplinary Research Figure 7. User registration form The registration form and validation is responsible for allowing users to input personal information. It ensures that all needed user information are present and the information are stored only once. However, it greatly suggested that the user must only use up to six characters and not to input an email account in the Username textbox. Users should only use a single name. When having two names like Excel Philip, It is suggested that user must use underscore or dash to make the single word only. Figure 8. Summary Registration Form Screen cast 3 is part of the user form registration. It summarizes and generates personal information of the user. It displays the user encoded Last Name, First Name, Middle Name, Course, Address, Username and Password. 57 International Peer Reviewed Journal Figure 9. User authentication form The User login form and validation form is responsible for accepting username and password. It ensures only the authorized users permitted by the professor to register in the system may take the exam and has not taken the exam yet. Otherwise, the software returns or displays on the screen the result of the exam. Figure 10. MCTR testing administration The testing and scoring is responsible for fetching all the questions that were encoded by the subject teacher. It systematically randomized the questions so that there will be no the same question posted on the screen. This effectively eliminates the issue of cheating among seatmates. It also ensures consistent and reliable scoring and recording. Also, another feature of MCTR that is present on this page is the reminder of the user of how many questions are still left to answer. The number of items decreases by one every time the user clicks the submit button. 58 JPAIR Multidisciplinary Research Figure 11. User exam result The Result Module is responsible for displaying the recorded score. It displays the correct number of items over the total number of items together with the corresponding percentile. Figure 12. User’s correct and incorrect answer The Result module shows both the correct and incorrect answers by the user per item. 59 International Peer Reviewed Journal Figure 13. Test results The Result Module shows the number of examinees who took the exam. Moreover, the software also prints the personal details arranged in an alphabetical manner, the number of correct items, the total number of times and the corresponding percentile score of all the users. While it is true that MCTR is secure, still, it is highly recommended that an updated anti-virus must be installed along with MCTR to ensure the integrity of the data stored in its database. The MCTR Usability The finding was drawn from the individual indicators which were all rated as Strongly Agree. The highest mean rating of 4.83 implies that the users strongly agree that the software helped them to be more effective. However the indicator “It gives me more control over the activities of my life” was rated lowest with the mean rating of 4.39 described as Usable. The finding supports the study of Adams, Nelson and Todd (1992) that usefulness is an important determinant of system use. The indicator “It is easy to use” was rated by the respondents as the highest with the mean rating of 4.85 described as Usable. It means that users find the software straightforward. On the other hand, the indicator “I don’t notice any inconsistencies as I use it” was rated lowest with the mean rating of 4.31 described as Usable which implies that the users found using the software reliably. The finding supports the study of Venkatesh and Davis (1996) that an individual’s perception of a system’s ease of use is attached to his general computer self-desire. 60 JPAIR Multidisciplinary Research Likewise, usability has an impact on ease of use perceptions about a system only after hands-on experience with the system. The indicator “I learned to use it quickly” was evaluated as the highest with the mean rating of 4.76 and described as Usable. This means that the respondents learned to use the software fast. While indicators “I easily remember how to use it” and “I quickly became skillful with it” were rated as lowest with the mean rating of 4.65 and described as Usable. This means that users were already “experts” in using the software even in a short span of time. This also means that the users were accustomed to computers that they can easily manipulate software. The Indicator “I am satisfied with it” was rated as the highest with the mean rating of 4.76 and described as Usable. This means that the respondents were contented. They are comfortable with the features, graphical user interface, and the user experience. On the other hand, the indicator “It is pleasant to use” was rated as the lowest with the mean rating of 4.7 and described as Usable. This means that users enjoyed while using the software. This may be because it was their first time using such type of examination in their school lives. The finding supports the study of Kekre, Krishnan and Srinivasan (1995) that capability and usability are the critical drivers of overall customer satisfaction. Client-Server Configuration MCTR is a web based application that provides service to client requests. One may use MCTR anytime. MCTR may be accessed anywhere within the servers’ reaching. Hardware requirements For the software to meet its best performance, it should have the proper hardware required. For the implementation of MCTR, the following computer hardware are needed:  Intel Celeron Processor G1820 (2M Cache, 2.70 GHz)  2GB DDR3 SDRAM  500GB   Gigabit Ethernet  Wireless router 61 International Peer Reviewed Journal Software requirements The following software should be installed for the successful implementation of the system:  xampplite for windows  Internet Browser (e.g. Mozilla Firefox) Limitations  Best viewed using Laptop and Desktop  Tablets and Smartphones (Ease of use is not guaranteed) CONCLUSIONS The study concludes that the students prefer computer-aided as the delivery method of assessment. Moreover, students prefer multiple choice type of test especially when the assessment period is limited. The existing testing administration is manual. The combination of Hypertext Mark-up Language (HTML), Cascading Style Sheet (CSS), and Hypertext Pre-processor (PHP) and MySQL relational database management system is deemed effective in the development of a Multiple Choice Test Randomizer. TRANSLATIONAL RESEARCH The software is intended for formative and summative types of test. However, it can be utilized in the administration of admission test and National Assessment Test. The software is beneficial especially when there is a great number of examinees because it automates the checking of test papers and provides test results immediately. LITERATURE CITED Adams, D. A., Nelson, R. R., & Todd, P. A. (1992). Perceived usefulness, ease of use, and usage of information technology: a replication. MIS quarterly, 227- 247. Retrieved on April 10, 2014, from http://goo.gl/ZV35BP Constantine, L. (2008). Computer-Aided Assessment: Design Considerations. Journal of Psychotherapy & the Family. Retrieved on April 7, 2014, from http://goo.gl/ZV35BP 62 JPAIR Multidisciplinary Research Fielding, A. (2003). Tools for Computer-Aided Assessment. Learning-Teaching Unit. Retrieved on April 7, 2014, from http://goo.gl/ZV35BP Georgia Online Assessment System. (2013, March 15). Retrieved on April 8, 2014, from http://goo.gl/ZV35BP Kekre, S., Krishnan, M. S., & Srinivasan, K. (1995). Drivers of customer satisfaction for software products: implications for design and service support. Management science, 41(9), 1456-1470. Retrieved on April 10, 2014, from http://goo.gl/ZV35BP Mitkov, R., LE AN, H. A., & Karamanis, N. (2006). A computer-aided environment for generating multiple-choice test items.  Natural Language Engineering, 12(02), 177-194. Sangwin, C. (2013). Computer aided assessment of mathematics. OUP Oxford. Retrieved on April 7, 2014, from http://goo.gl/ZV35BP Sheader, E., Gouldsborough, I., & Grady, R. (2006). Staff and student perceptions of computer-assisted assessment for physiology practical classes. Advances in Physiology Education, 30(4), 174-180. Retrieved on April 8, 2014, from http://goo.gl/ZV35BP Venkatesh, V., & Davis, F. D. (1996). A model of the antecedents of perceived ease of use: Development and test*. Decision sciences, 27(3), 451-481. Retrieved on April 9, 2014, from http://goo.gl/ZV35BP