International Journal of Interactive Mobile Technologies (iJIM) - Volume 4, Issue 4, October 2010 MOBIQIYAS: A MOBILE LEARNING STANDARDIZED TEST PREPARATION FOR SAUDI ARABIAN STUDENTS MobiQiyas: A Mobile Learning Standardized Test Preparation for Saudi Arabian Students doi:10.3991/ijim.v4i4.1446 Mohssen M. Alabbadi Computer Research Institute (CRI), King Abdulaziz City for Science and Technology (KACST), Riyadh, Saudi Arabia Abstract—A mobile learning system, called MobiQiyas, for preparing Saudi Arabian students for one of the standard- ized tests, given by the National Center for Assessment in Higher Education (NCAHE), has been developed, using ready-made commercial products and tools. The learning material of MobiQiyas consists of practice questions with their answers, both provided by NCAHE, to be loaded by the students into their own mobile phones; after installation, the students can interact with MobiQiyas any number of times, as desired, without incurring any additional cost, other than the initial airtime cost for downloading. From total number of students taken the test, 20,000 students were randomly selected to use MobiQiyas and information was collected from them to measure their attitudes and partici- pation of MobiQiyas. It was found that 36.1% of students had actually downloaded MobiQiyas successfully. Further- more, a telephone survey was conducted, after the test pe- riod, on a class of 40 students in a secondary school in Ri- yadh, taking the same test, to measure their acceptance of MobiQiyas, using a 9-item questionnaire based on a 5-point Likert scale. The responses of the 40 students reflected high acceptance and satisfaction levels of MobiQiyas as an effec- tive test prep tool. Index Terms—Educational technology, Mobile learning (mLearning), Test preparation (Test Prep), Standardized test, Technology acceptance, User interfaces. I. INTRODUCTION The lowering cost of mobile devices and the availability of wireless infrastructures have created the “anytime, anywhere, on-the-move” paradigm. The mobile phones market, producing 1.2 billion new mobile phones annually [9], has exceeded the laptop shipment since 2006 [24], where new capabilities in terms of hardware and software are introduced in each new release, with the ability, avail- able only on several models now, to easily acquire and install 3rd-party applications. These devices are thus turned into indispensable ubiquitous tools that could replace the desktop and laptops1. This new paradigm offers powerful features and func- tions such as mobility, reachability, localization, flexibil- ity, information accuracy, and motivational effects and productivity enhancement due to self controlling and bet- ter use of spare time. Mobile-based applications are radically transforming the way people communicate, access and utilize information resources, and connect with 1 The survey, conducted by the Pew Internet & American Life Project, predicted that most people across the world will be using a mobile device as their primary means for connecting to the Internet by the year 2020[5]. peers and colleagues. Mobile-based applications were first employed within gaming, movies, or other sectors of the entertainment industry, taking about 12-18 months for their adoption into mainstream industry [7]. They have established a new dimension for providing services such as mobile commerce (mCommerce), mobile business (mBusiness), mobile banking (mBanking), mobile health (mHealth), etc. The Wholesale Applications Community (WAC)2 will definitely increase the overall market for mobile-based applications. There was an impetus to use this new paradigm in the learning environment, thus creating “mobile learning,” m- learning or mLearning for short, with expected benefits to be reflected in more efficient and improved learning re- sults. These opening opportunities of mLearning are fac- ing, of course, some challenges and questions. Various issues, particularly user-interface related issues, have been raised, due to weaknesses of the mobile devices such as very small screen displays, low resolution, low processing power, restricted input capabilities of some of these de- vices, and limited storage capability, thus making the vi- ability of mobile technology in learning questionable. These issues play an important role toward the implemen- tation of mLearning. Therefore, it is critical to study the learner's adoption and acceptance of mLearning for some special application or applications. Test preparation (i.e., test prep), in particular for stan- dardized tests3, is very important learning activity to al- most all students. Students all over the world are taking 2 WAC (http://www.wholesaleappcommunity.com/default.aspx), a not-for- profit open global alliance, was created on Feb. 2010, by some of the world’s leading telecommunications operators and device manufactur- ers, fully supported by the GSM Association (GSMA). It aims to estab- lish an open ecosystem to spur the creation of mobile-based applica- tions, regardless of device, operating system (OS), or operator, by en- couraging open standardized technologies and providing complimentary commercial models to allow paying the developers for the applications sold through any associated application store. 3 The earliest record of standardized testing comes from China, taking by applicants for government jobs [6] for the imperial examina- tions and it was then institutionalized during the 6th century CE, under the Sui Dynasty [22]. On the contrary, examiners in the Western world favored essays, a tradition stemming from the ancient Greeks' affinity for the Socratic methodology [6]. During and after the Industrial Revo- lution, standardized tests had emerged in the Western world as an easy way to test large numbers of students quickly [6]. By World War I, standardized testing was a common practice in the Western World. Grading of standardized tests was at first done manually, undermining standardized testing's goal of speedy mass assessment, until the IBM 805 Test Scoring Machine, considered to be the first automatic test scanner, was developed in 1938 [8], where electrical current was used to detect marks made by special pencils (e.g., simple No. 2 pencils) on the answer sheets, giving rise to the bubbling-in of answers [6]. 4 http://www.i-jim.org http://dx.doi.org/10.3991/ijim.v4i4.1446� MOBIQIYAS: A MOBILE LEARNING STANDARDIZED TEST PREPARATION FOR SAUDI ARABIAN STUDENTS more standardized tests than ever before, and at ages long before entering colleges. Standardized tests have become one of the largest determining factors in the college- admissions process, particularly for élite schools. A standardized test is a test in which the same test is given in the same manner to all test takers such that its administration and scoring are done in a predetermined consistent manner, thus allowing more reliable compari- son of outcomes across all test takers [22]. Standardized tests are often time-limited tests composed of multiple- choice and true-false questions because they can be given and scored inexpensively. However, standardized tests can nearly use any other form of assessment such as essay questions and fill-in-the-blank analogies (e.g., grass:green::sky:____), which both require human evalua- tors. The TOEFL® (Test of English as a Foreign Lan- guage™) test, provided by the Educational Testing Ser- vice (ETS) [http://ets.org/], the SAT® (Scholastic Aptitude Test™) test, provided by the College Board (http://www.collegeboard.com/), and ACT® (American College Testing), provided by ACT, Inc. (http://www.act.org/) are some examples of standardized tests. In Saudi Arabia, the National Center for Assess- ment in Higher Education (NCAHE), subsidiary of the Ministry of Higher Education (MoHE), known as Qiyas4, (http://www.qeyas.com/Qiyas/Info/English.aspx), is re- sponsible for preparing, administrating, and giving stan- dardized tests. The tests include 9 educational tests and 3 vocational and licensing tests. In general, test prep is an activity, where the students normally study in isolation, without the immediate sup- portive environment, and become aware of non- understood aspects or concepts, thus making the students accept the extra charges occurring with mLearning. Fur- thermore, the mLearning features of motivation and self- paced learning are fully pronounced in test prep, allowing the students to utilize their spare time effectively. In this research, a test prep system based on mLearning, called MobiQiyas, is designed and developed, using ready-made commercial products and tools, for one of the standardized tests given Qiyas in Saudi Arabia, on the period of May 7-14, 2009. After MobiQiyas is downloaded and installed into the student's mobile phone, the student can practice solving questions that were given in previous tests of Qiyas, any number of times, as desired by the student, without incurring any additional cost on the student, other than, of course, the initial airtime cost for downloading. For the purpose of this initial trial of MobiQiyas, 20,000 students of the total number of stu- dents taking the test were randomly selected to use Mo- biQiyas. Information was collected from the 20,000 ran- domly-selected students and another 40 students who be- longed to one class of a secondary school in Riyadh for the purpose of measuring their attitudes, participation, and acceptance of MobiQiyas. The structure of the rest of this paper is as follows. Sec- tion 2 defines mLearning and describes its potentials, whereas Section 3 discusses the use of mLearning for test prep, looking at its trends and some of the products avail- able in the market. The description of MobiQiyas is given in Section 4 followed by its trial in Section 5, where the responses of the students, to measure their attitudes, par- 4 Qiyas is an Arabic word, meaning “measure.” ticipation, and acceptance, are analyzed. Finally, the con- cluding remarks are provided in Section 6. II. MOBILE LEARNING Mobile learning can be defined as any service or facility for knowledge transfer of events, content, tools, and applications to the learner [4], regardless of location and time [11], resulting in learner's alteration in behaviour [7], where mobile handheld devices (e.g., mobile phones, personal digital assistants (PDAs), and smart phones) are used, while the learner, but not necessarily the learning material providers, could be on the move. The behaviourist requirement indicates that learning is not deemed without the learner's alteration in behaviour [7]. Furthermore, the use mobile handheld devices, possibly on the move, excludes the use of labtops from mLearning [4], thus limiting mLearning to those wireless communi- cation devices that can be used while on-the-move [10,14]. Learning activity is a dynamic activity that can be closely linked to mobility with respect to space, time, and topic areas [21], making a perfect match with mLearning ─ learning occurs at different places (e.g., learning institutes, workplaces, homes, and even places of leisure), at different times (e.g., working days, weekends, or holidays), and between different topic areas of life (e.g., education, work, self-improvement, or leisure) [21]. The diversity of space for adults' daily self-learning was shown5 in [18], thus opening more opportunity for mLearning to support learning during the growing amounts of time that people spend on the move. Mobile learning is on the intersection of mobile computing and e-learning [13,19], conveying e-learning through mobile devices using wireless connectivity; this intersection includes the use of desktop as well as labtops. It breaks the constraints of time and space, which have become a very important requirement of E-Iearning, thus constructing a flexible and open learning environment. This environmen can provide access, context, and collaboration, and supply additional facilitation measures for facilitators [7]. MLearning actively engages learners, emphasizing learner centeredness to match all learners' styles of learn- ing [14]. From an activity-centered perspective, the six existing learning theories (i.e., behaviourist, constructivist, situated, collaborative, informal and lifelong learning theories) can be harnessed by applying mLearning new practices [15]. Furthermore, the ability to record informa- tion about new encountered experiences, using the en- hanced features on the mobile devices, enables experien- tial learning [14]; according to experiential learning the- ory, ideas and concepts are not fixed, but are formed and modified through the experiences the learners have and by their past experience. Generally speaking, mLearning systems can be divided into three types: push-based, application-based, or browser-based mLearning systems [12]. The push-based systems use the mobile phone email or Short Message 5 The daily self-learning episodes for adults took place 51% at the learner’s usual environment (i.e., learner’s home or office), 21% in the workplace outside the office, 14% in other locations (such as places of worship, the doctor’s office, cafes, hobby stores, and cars), 6% at places of leisure, 5% outdoors, 2% in a friend’s house, and only 1% on transport[18]. iJIM – Volume 4, Issue 4, October 2010 5 http://ets.org/� http://www.collegeboard.com/� http://www.act.org/� http://www.qeyas.com/Qiyas/Info/English.aspx� MOBIQIYAS: A MOBILE LEARNING STANDARDIZED TEST PREPARATION FOR SAUDI ARABIAN STUDENTS Service (SMS), whereas the browser-based systems re- quire an Internet-enabled mobile device, using HTML or wireless application protocol (WAP). Table I gives a comparison of these systems, summarizing their advan- tages and disadvantages. These systems have been used, either as a single system or as a combination of two or more of them, in some k-12 schools and universities or in career development area, for class learning as well in out- door learning. TABLE I. COMPARISON OF MLEARNING SYSTEMS System Advantage Disadvantage Push-based Low cost Long interactive time; Poor and limited con- tent (for SMS) Application- based Moderate cost Device Dependent Browser-based Rich content High cost The 2009 Horizon Report identified mLearning as one of the emerging technologies, with large impact on teach- ing, learning, research, or creative expression, to be adopted into the mainstream of learning-focused organiza- tions within the next year (i.e., the first adoption horizon, as defined by Horizon) [9]; Indeed, mLearning was identi- fied at different adaption horizons in the previous two Horizon annual reports [9]. Ambient Insight indicated that the 2nd generation of mLearning will start in 2011, which is characterized as mobile collaboration; the 1st generation, already started in 2007 and to be ended by 2010, includes the three major “native” types of mLearning: location- based mLearning, device-embedded mLearning, and handheld decision support [4]. Ambient Insight indicated that the US mLearning mar- ket reached US$538 million and US$632.2million in 2007 and 2009, respectively and it predicts it to continue to grow in the US by a 5-year compound annual growth rate (CAGR) of 18.3%, where revenues will reach US$1.4 billion by 2014 [1]. III. STANDARDIZED TEST PREP BASED ON MLEARNING As of 2009, the traditional US academic classroom- based test preparation industry was a US$3.1 billion in- dustry, excluding exam preparation revenues related to State-mandated K-12 testing, where the two largest sup- pliers, accounting for 45% of all revenues, are the Prince- ton Review (http://www.princetonreview.com/) and Kap- lan (http://www.kaptest.com/), owned by the Washington Post [4]. However, due to decline in revenues, the class- room-based test prep industry is trying to move to tech- nology-based learning products. The market of technol- ogy-based learning products, for test prep, compliance assurance, skills assessment, and NCLB6-related testing, is expected to grow by 20-22%, due to consumer, corpo- rate, government, and academic demand [4]. Standardized test prep based on mLearning systems are either push-based or application-based systems; the high cost of browser-based systems makes them unattractive for test prep at the present time. However, the browser- based systems has the advantage of simply implementing 6 President George W. Bush's 2001 No Child Left Behind edu- cation reform demanded the use of state-mandated standardized testing as a means of assessing school performance[6]. adaptability by providing test prep tailored to the learner's ability level, in a similar way as in computerized adaptive testing (CAT). These systems are called computerized adaptive test on mobile devices (CAT-MD) [20]. A proto- type of a CAT-MD, using PDAs was designed and devel- oped and evaluated positively by users as well as some experts [20]. An example of push-based mLearning system for stan- dardized test prep is smsPREP (http://smsprep.com/), from smsPREP, Inc. It uses SMSs to supply its US sub- scribers, for US$19.99 per month, with practice questions, flashcards, and test-taking strategies, for the PSAT, SAT, ACT tests. The subscriber of smsPREP simply chooses the subject (i.e., math, critical reading, writing, or strate- gies) and the preferred time, then the correspondence in- formation is sent by SMSs to the subscriber's specified telephone number; for practice questions, the answers and their explanations are sent to the subscriber by SMSs, after the subscriber answers the questions through SMSs. Furthermore, an SMS-based collaborative mLearning system, called the Dynamic Frequently Asked Questions (DFAQ), was designed and developed at the University of Cape Town (UCT) in South Africa [16], which can be used as a consultation tool during test prep. DFAQ, a seamless web-based application with an SMS interface, allows the students to post questions anonymously, from their own personal study environments, using SMS mes- sage, and receive responses, through SMS message from anonymous sources (i.e., peers, educators, and/or tutors) unless otherwise stated. The high increase in the use of the SMS feature of the DFAQ, by almost 200% as compared during regular class time [16], reflects the need of the stu- dents for technology-based test prep tools, thus showing their effectiveness. The limitations of SMS message on the size7 (i.e., 160 characters per message) and the type of information (i.e., only text) make application-based systems the most attrac- tive solution now. There are mobile applications for stan- dardized test prep available in the market that can be sim- ply bought from mobile application stores such as App Store8 for the Apple iPhone. Kaplan Inc. teamed with Jirbo Inc. to release “Kaplan SAT Flashcubes,” for iPod Touch and iPhone, including self-study mode and multiple-choice quiz mode, with the capability to track the learner's progress; it is available on the iTunes store (http://itunes.apple.com/app/kaplan-sat flashcubes/id300615175), for US$4.99, updated on Jan. 30, 2009. The Princeton Review, on the other hand, joined with Modality, Inc. to release, the “GRE Vocab Challenge App” (http://modality.com/apps/GRE-Vocab-Challenge- by-The-Princeton Review_p_139.html) for iPhone and iPod touch; it can be purchased through, for US$4.99, through the App Store. In addition, some standardized test providers are pro- viding mLearning test prep systems. For example, ETS, on December 16, 2008, joined with Nokia to launch Eng- lish learning courses, designed by ETS teaching and test- ing experts, for Nokia phone users in China 7 This has led to conciseness in messages, producing a discur- sive type of SMS lingo. SMS lingo has become an acceptable SMS language among mobile phone users and there are several SMS lingo dictionaries[23]. 8 In mid-2008, Apple launched the App Store and less than six months later, more than 10,000 such applications were offered. 6 http://www.i-jim.org http://www.princetonreview.com/� http://www.kaptest.com/� http://smsprep.com/� http://itunes.apple.com/app/kaplan-sat� http://modality.com/apps/GRE-Vocab-Challenge-by-The-Princeton Review_p_139.html� http://modality.com/apps/GRE-Vocab-Challenge-by-The-Princeton Review_p_139.html� MOBIQIYAS: A MOBILE LEARNING STANDARDIZED TEST PREPARATION FOR SAUDI ARABIAN STUDENTS http://ets.org/newsroom/news_releases/ets_nokia_learning _english. The courses can be downloaded using Nokia’s Mobiledu (http://www.mobiledu.cn/en/index.html), where the available courses include TOEFL-type practice ques- tions and TOEIC-type practice questions. IV. MOBIQIYAS MobiQiyas is a mobile learning system for preparing the students for the tests given by Qiyas. It was designed and developed using ready-made commercial products and tools. MobiQiyas provides “anytime, anywhere, and the move” resources with rich interaction for effective learning and performance-based assessment. In addition, it is designed to produce support, motivation, introductions, tips, revision, and study guides. MobiQiyas was motivated by the fact that all students taking the tests of Qiyas own or have access to mobile phones. Upon receiving an SMS message, sent automatically by a local SMS provider, on the student's mobile phone. The student then simply clicks on the link provided by the SMS to access MobiQiyas as shown in Figure 1. The learning material of MobiQiyas is structured into modules, where each module pertains to one of the tests provided by Qiyas; there could be more than one module related to one single test. Each module consists of questions, provided by Qiyas along with their answers; the questions were asked in pre- vious tests of Qiyas. The fact that the questions and their answers were provided by Qiyas and the questions were used in previous tests of Qiyas enhances the credibility of MobiQiyas in the eyes of students. Each module of MobiQiyas is an application based on Java ME (Java Platform, Micro Edition), making it port- able across almost all mobile phones as well embedded devices such as personal digital assistants (PDAs), TV set- top boxes, etc. It was developed using the Learning Mo- bile Authoring (LMA)9. The LMA enables the creation, customization, review, and update of rich interactive con- tent, containing text, images, audio, and/or video. For MobiQiyas, the content contains neither audio nor video. However, images are used extensively for the inclusion of mathematical equations, special symbols, figures, tables, chemical compounds, etc., providing MobiQiyas with high quality display, thus imitating the real paper-based tests. The questions are true-false or multiple-choice ques- tions but MobiQiyas has the capability to provide blank filling questions which were never used in Qiyas's tests. For the multiple-choice questions, the correct answers are automatically fed back to the student after two unsuccess- ful trials; the number of trials is a system variable and could be changed accordingly by the system designer. Each module is equipped with a table of contents and navigational capabilities, using user-friendly icons such as “home,” “next,” etc. MobiQiyas has the capability to provide not only the correct answers but also the solutions of the questions and references for further reading. In addition, it can also sup- ply introductory material, review, and study guides. 9 LMA is a free software available from Hot Lava Software, acquired by OutStart in 2009, available at http://www.outstart.com/hot- lava-download-registration.htm. After the student downloads the desired module into his/her mobile phone and the module is installed, the stu- dent can then interact with the questions any number of times, as desired, without incurring any additional cost on the student, other than, of course, the initial airtime cost for downloading the desired module. V. MOBIQIYAS IN ACTION MobiQiyas was launched on 30 April, 2009, lasted until 14 May, 2009, to prepare the students for The General Capabilities Test (GCT), given by Qiyas on the period 7- 14 May, 2009. GCT, containing verbal and quantitative sections, is mandatory for all graduates of secondary schools, before applying for higher education, vocational, or military institutes. For the GCT test, MobiQiyas con- tained 50 practice questions. Figure 2 shows the table of contents of MobiQiyas of this GCT. In the Appendix, snapshots of some questions for MobiQiyas of this GCT are shown. For the purpose of this initial trial of MobiQiyas, 20,000 students were selected randomly, across Saudi Arabia, from the total number of students, who were tak- ing this particular GCT test, to measure their responses to MobiQiyas. Furthermore, the Mobile Delivery and Track- ing System (MDTS), from Hot Lava Software (acquired by OutStart in 2009), a WAP-based environment, was used to track and record students' responses and for the delivery and management of the GCT test prep module. Each of the 20,000 randomly selected students received an SMS message, sent automatically by a local SMS pro- vider on 30 April, 2009. The report produced by the SMS Figure 1. MobiQiyas Figure 2. Table of content of MobiQiyas for the GCT test of Qiyas iJIM – Volume 4, Issue 4, October 2010 7 http://ets.org/newsroom/news_releases/ets_nokia_learning_english� http://ets.org/newsroom/news_releases/ets_nokia_learning_english� http://www.mobiledu.cn/en/index.html� MOBIQIYAS: A MOBILE LEARNING STANDARDIZED TEST PREPARATION FOR SAUDI ARABIAN STUDENTS provider indicated that the message delivery failure ratio was 2.2%, thus the number of students who actually re- ceived the SMS message was 19560. The SMS message, shown in Figure 4, mentioned that the student was se- lected for the MobiQiyas trial and it further provided in- formation for the student for setting up his/her mobile for Internet connection, as specified by his/her carrier. It also provided information about the cost for downloading the test preparation module; the carrier's airtime-cost was about 1.5 Saudi Riyal (US$0.40). Furthermore, the SMS message supplies a link to be clicked, if the student desires to download the GCT test prep module. For the purpose of this initial testing of MobiQiyas, the attitudes, participation, and acceptance of MobiQiyas of the students are measured. The information for the stu- dents' attitudes and participation are collected automati- cally using MDTS; this is can be viewed as the online mode of MobiQiyas, where the students were interacting online with MobiQiyas. On the other hand, the interaction of the students with the questions of module, after downloading and installing the module, can be considered as the offline mode of MobiQiyas. In the offline mode, the student has the freedom to interact with test prep module any number of times, as desired, without incurring any additional cost, other than, of course, the initial airtime cost. The mixing of these two modes reduces the cost in- curred on the students. It is to be emphasized that the characteristic of the sample space for this trial (i.e., ran- dom selection of students from secondary schools) limits the amount of collected information from the students. A. Students' attitudes and participation A survey was conducted on the students, consisting of two parts. The first part was before downloading the GCT test prep module, which is a questionnaire of two simple questions, whereas the second one was after downloading the test prep module, consisting of only one question. The survey measured the students' attitudes and participation for MobiQiyas. The survey, however, allows the student to skip any of the asked questions by clicking on the "next" link. This option was intentionally chosen to meas- ure students' behavior and their tendencies to participate on surveys. The responses of the students were tracked and re- corded by the MDTS as given in Table II. The first ques- tion is about students' readiness, where only 55% of them indicated that they used the Internet on their mobile phones. It is expected that percentage should be higher than that, considering AdMob Mobile Metrics Report, on July 2009, where Saudi Arabia was ranked 7th in the world, with 151,917,199 downloads of mobile applica- tions [3]. For the downloading of the GCT test prep module, 65% of the students, who accessed MobiQiyas, indicated that they successfully downloaded the module. On the other hand, accessing MobiQiyas and the download of the mod- ule were automatically tracked and recorded by the MDTS, as depicted in Figure 4. As shown in Figure 5, the number of students who ac- tually accessed MobiQiyas is 7060 students. They all had downloaded the GCT test prep successfully, representing 7060/19560 = 36.1% of the students who actually re- ceived the SMS message. Figure 3. The SMS message sent to the 20,000 randomly selected students Figure 4. The number of students who accessed MobiQiyas and downloaded the GCT test prep successfully as recorded automatically by the MDTS. TABLE II. RESPONSES OF STUDENTS Question Yes No Skipped 1 Have you ever accessed the Internet through your mobile phone? 55% 40% 5% 2 Do you want to download the test prep module? 94% 6% 3 Did you download the test prep module successfully? 65% 31% 4% Examining the students' responses to the 3rd question in Table II and Figure 4 shows a major discrepancy. This could be attributed to students' confusion between downloading and installation, even though there was a message after the download, indicating that the download was successful. Clearly, the data in Table II reflects, in general, the ten- dency of the students, in particular high school students, to participate non-positively in surveys, in non-controlled environment. This may be attributed to students' fear of consequences, making the students to skip the questions and giving inaccurate responses. This suggests using an- 8 http://www.i-jim.org MOBIQIYAS: A MOBILE LEARNING STANDARDIZED TEST PREPARATION FOR SAUDI ARABIAN STUDENTS other method to measure students' acceptance of MobiQi- yas. Students' acceptance should be measured in a con- trolled environment. B. Students' Acceptance After the period of the test was over, another survey was conducted on a class of 40 students belonging to one class of a secondary school in the city of Riyadh, where the students had taken the same test as the 20,000 ran- domly selected students; fortunately, the students of this class were not part of the 20,000 randomly selected stu- dents. This is a telephone survey performed by one of the 40 students. The telephone was used to conduct the survey because most of the students were in vacation and tele- phone was the best way to reach them. Furthermore, using telephone provides a level of seriousness, which is mostly needed for this particular age of the students. For this trial, Shackel's usability model [17] is one of the most suitable methods to measure the acceptance of learners, since it considers usability to be an aspect that influences product acceptance. With this model, for a sys- tem to be usable, it has to achieve defined levels on the following four constructs: effectiveness, learnability, flexibility, and attitude [17]. The nature of this survey prevents us from using lengthy questionnaire, as was done in [2]. A 9-item questionnaire was designed, where the ques- tions, to some extent, can be mapped to the questions in [2]. The survey is based on a 5-point Likert scale, ranging from “Strongly Disagree” to “Strongly Agree.” The re- sponse that indicates the lowest approval (i.e., “Strongly Disagree”) received a score of 1, with an increase of 1 point for each response (i.e., 2 points for “Disagree,” 3 points for “Neutral,” 4 points for “Agree,”) until the re- sponse that indicates the greatest approval (i.e., “Strongly Agree”) received a score of 5. This instrument has thus maximum score of is 5*9=45 and minimum score is 9. Data gathered on this questionnaire were coded in SPSS for analysis purposes. The responses of the students to the questionnaires are summarized in Table III. TABLE III. TYPE SIZES FOR CAMERA-READY PAPERS Question Mean Standard Deviation 1 Practicing MobiQiyas was very enjoyable. 4.38 .91 2 I think all my friends who used MobiQiyas are happy about it. 4.13 .87 3 I would like to use a similar system for other test preparation. 4.20 .92 4 Practicing MobiQiyas was effec- tive. 4.20 .73 5 Practicing Using MobiQiyas has motivated me. 4.37 .92 6 MobiQiyas made me better use my leisure time. 4.33 .88 7 MobiQiyas helped me to learn new words. 4.31 .82 8 MobiQiyas helped me to learn new concepts. 3.11 1.82 9 MobiQiyas helped me to correct some misunderstood concepts. 4.10 .81 Scoring: 5 = Strongly Agree, 1 = Strongly Disagree As it is clearly shown in the table, the students' accep- tance of MobiQiyas is very high, where most responses score more than 4, implying “Agree” response. There was only one question with a score of 3.11, regarding whether the student learned new concepts from MobiQiyas. This score is very reasonable, considering that MobiQiyas in this trial only supplies practice questions. In general, the students' responses reflect high acceptance and satisfac- tion levels for MobiQiyas as a test preparation tool. Fur- thermore, the students expressed their desires to have more questions and their willingness to pay for the in- curred charges. C. Students' Comments After launching MobiQiyas on April 30, 2009, a press was released by Qiyas in most major newspapers in Saudi Arabia, describing the service. The comment sections, related to the press release, of these newspapers, were monitored. Most of the comments came from students who were not randomly selected by Qiyas, showing inter- est to be part of the trial. Indeed, some of them asked to publish the link; this shows the readiness of the students to use MobiQiyas. One interested comment stated that “this is the real meaning for eLearning.” On the contrary, there were some negative feedbacks, concerning only the cost of MobiQiyas. VI. CONCLUSION The mLearning-based test prep, called MobiQiyas, for preparing students for one of the standardized tests given by Qiyas in Saudi Arabia, provides learning material, con- sisting of practice questions with their answers. MobiQi- yas is to be loaded to the student's mobile phones and after installation, the students can interact with it any number of times, as desired by the student, without incurring any additional cost, other than the initial airtime cost for download. MobiQiyas was put into action with 20,000 students, selected randomly from the total students taken the test, where 36.1% of students had actually accessed and downloaded it. After the test period, a telephone sur- vey was conducted on only 40 students to measure their acceptance of MobiQiyas using a 9-item questionnaire based on a 5-point Likert scale, ranging from “Strongly Disagree” to “Strongly Agree.” The responses from those 40 students reflect high acceptance and satisfaction levels of MobiQiyas as a test preparation tool. ACKNOWLEDGMENT The author would like to thank the National Center for Assessment in Higher Education (NCAHE) for providing the questions and their answers. In addition, special thanks are due to Mr. Mohammed Alabbadi for conducting the telephone survey and Mr. Hasan Alabbadi for his help in generating the learning material of MobiQiyas. REFERENCES [1] S. S. Adkins, “The state of mobile learning industry,” mLearn- Con: The Premier mLearning Conference & Expo, June 15-17, 2010, San Diego, USA (Presentation of the key find- ings of the Ambient Insight's report, “The US Market for Mobile Learning Products and Services: 2009-2014 Forecast and Analy- sis,” Monroe, WA, USA: Ambient Insight, LLC; Retrieved on Sep. 9, 2010, from the following URL: http://www.ambientinsight.com/Resources/Documents/Ambient- Insight-Mobile-Learning-Market-2009-2014.pdf). iJIM – Volume 4, Issue 4, October 2010 9 http://www.ambientinsight.com/Resources/Documents/Ambient-Insight-Mobile-Learning-Market-2009-2014.pdf� http://www.ambientinsight.com/Resources/Documents/Ambient-Insight-Mobile-Learning-Market-2009-2014.pdf� MOBIQIYAS: A MOBILE LEARNING STANDARDIZED TEST PREPARATION FOR SAUDI ARABIAN STUDENTS [2] AdMob, “AdMob Mobile Metrics Report”, July 2009 (Retrieved on Sep. 9, 2010, from URL:http://metrics.admob.com/wp- content/uploads/2009/08/AdMob-Mobile-Metrics-July-09.pdf). [3] M. M. Alabbadi, “Learners' acceptance based on Shackell's usabil- ity model for supplementary mobile learning of an English course, ” In the Proceedings of the 2nd International Conference on Com- puter Supported Education (CSEDU2010), April 7-10, 2010, Va- lencia, Spain. [4] Ambient Insight, “Ambient Insight’s 2009 Learning and Perform- ance Technology Research Taxonomy,” Monroe, WA, USA: Am- bient Insight, LLC, September 2009 (updated) (Retrieved on Sep. 9, 2010, from URL: http://www.ambientinsight.com/Resources/Documents/AmbientIn sight_Learning_Technology_Taxonomy.pdf). [5] J. Anderson and L. Rainie, “The Future of the Internet III,” Pew/Internet: Pew Internet & American Life Project, Washington, DC, USA: Pew Internet & American Life Project, Dec. 14, 2008 (Retrieved on Sep. 9, 2010, at URL: http://www.pewinternet.org/~/media//Files/Reports/2008/PIP_Fut ureInternet3.pdf.pdf). [6] D. Fletcher, “Standardized testing,” TIME, Dec. 11, 2009. Re- trieved online, on Sep. 9, 2010, at the following URL: http://www.time.com/time/nation/article/0,8599,1947019,00.html. [7] S. J. Geddes, “Mobile learning in the 21st century: Benefit for learners,” Knowledge Tree E-journal, Edition 6, 2004 (Retrieved on Sep. 9, 2010, from the following URL: http://knowledgetree.flexiblelearning.net.au/edition06/download/G eddes.pdf). [8] IBM Archives, Exhibits, IBM Special Products (Vol. 1), “The IBM 805 test scoring machine,” New York, NY, USA: IBM Cor- poration (Retrieved on Sep. 9, 2010, at the following URL: http://www03.ibm.com/ibm/history/exhibits/specialprod1/specialp rod1_9.html). [9] L. Johnson, A. Levine, and R. Smith, “The 2009 Horizon Report,” Austin, Texas, USA: The New Media Consortium (NMC), 2009 (joint collaboration between NMC and the EDUCAUSE Learning Initiative (ELI), an EDUCAUSE Program; Retrieved on Sep. 9, 2010, at URL http://www.nmc.org/pdf/2009-Horizon-Report.pdf). [10] D. Keegan, “ The incorporation of mobile learning into main- stream education and training,” In the 4th World Conference on mLearning (mLearn 2005), Cape Town, South Africa, October 25- 28 2005 (Retrieved on Sep. 9, 2010, at URL: http://www.mlearn.org.za/CD/papers/keegan1.pdf). [11] F. Lehner, H. Nösekabel, and Lehmann H. “Wireless E−learning and communication environment: WELCOME at the University of Regensburg,” In The Proceedings of the First International Work- shop on M-Services - Concepts, Approaches, and Tools (ISMIS'02), Lyon, France, June 26, 2002, CEUR-WS.org, CEUR Workshop Proceedings, Vol-61, Z. Maamar, W. Mansoor, and W. van den Heuvel, Eds. (Retrieved on Sep. 9, 2010, at URL: http://ftp.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol- 61/paper2.pdf). [12] Y. Li, H. Guo, G. Gao, R. Huang, and X. Cheng, “Ubiquitous e- learning System for Dynamic Mini-courseware Assembling and Delivering to Mobile Terminals,” In the Proceedings of the Fifth International Joint Conference on INC, IMS, and IDC (NCM 2009), August 25-27, 2009, Seoul, Korea, J. Kim, D. Delen, Park J., F. Ko, C. Rui, J. H. Lee, W. Jian, and G. Kou, Eds. Los Alami- tos, CA, USA: IEEE Computer Society’s Conference Publishing Services (CPS), 2009, pp. 1081-1086. [13] M. Milrad, “Mobile learning: Challenges, perspectives, and real- ity,” Mobile Learning Essays on Philosophy, Psychology and Education, K. Nyiri, Ed. Vienna: Passagan Verlag, 2003, pp. 151- 164. [14] P. B. Muyinda, E. Mugisa, and K. Lynch, “M-Learning: The edu- cational use of mobile communication devices,” In the Proceed- ings of the 3th Annual International Conference On Computing and ICT Research (SREC 07), Volume III, J. M. Kizza, J. Mu- hirwe, J. Aisbett, K. Getao, V. W. Mbarika, D. Patel, A. J. Rodri- gues, Editors, Kampala, Uganda, August 5-8, 2007, pp. 290-301, Fountain Publishers: Kampala, Uganda, 2007 (Retrieved on Sep. 9, 2010, at URL: http://cit.mak.ac.ug/iccir/downloads/SREC_07/Paul%20Birevu%2 0Muyinda%20,%20Ezra%20Mugisa%20,%20Kathy%20Lynch_0 7.pdf). [15] L. Naismith, P. Lonsdale, G. Vavoula, and M. Sharples, “Litera- ture review in mobile technologies and learning,” NESTA Future- lab Series, Report 11, NESTA Futurelab: Bristol, UK, 2004. (Re- trieved on Sep. 9, 2010 at URL: http://elearning.typepad.com/thelearnedman/mobile_learning/repo rts/futurelab_review_11.pdf; also available at URL: http://www.futurelab.org.uk/resources/documents/lit_reviews//Mo bile_Review.pdf.). [16] D. Ng’ambi and A. Knaggs, “Using mobile phones for exam preparation,” In the Proceedings of the IADIS International Mo- bile Learning Conference, Algarve, Portugal. 11-13 April. 2008, pp. 35-42. [17] B. Shackel, “Usability – Context, Framework, Design and Evalua- tion,” In B Shackel, B. and Richardson, S. (Eds.), Human Factors for Informatics Usability, Cambridge, UK: Cambridge University Press, 1991, pp. 21-38. [18] M. Sharples, J. Taylor, and G. Vavoula, “Towards a theory of mobile learning," In the Proceedings of the 4th World Conference on Mobile Learning (mLearn 2005), 25-28 October 2005, Cape Town, South Africa (Retrieved on Sep. 9, 2010, at URL: http://www.mlearn.org.za/CD/papers/Sharples%20Theory%20of% 20Mobile.pdf). [19] A. Stone, Designing “Scalable, effective mobile learning for mul- tiple technologies,” In Learning with Mobile Devices, J. Attwell and C. Savill-Smith, Eds. London, UK.: Learning and Skills De- velopment Agency, 2004. [20] E. Triantafillou, E. Georgiadou, and A. A. Economides, “The design and evaluation of a computerized adaptive test on mobile devices,” Computers & Education, Volume 50, Issue 4, May 2008, pp. 1319–1330 (Available online, as of Sep. 08, 2010, at www.sciencedirect.com). [21] G. N. Vavoula and , M. Sharples, “KLeOS: A personal, mobile, knowledge and learning organisation system,” In the Proceedings of the IEEE International Workshop on Wireless and Mobile Technologies in Education (WMTE 2002), August 29–30, 2002, Växjö, Sweden, M. Milrad and U. H. Kinshuk, Eds. Los Alamitos, CA, USA: The IEEE, Inc., 2002, pp. 152-156. [22] Wikipedia, the free encyclopedia, “Standardized test,” Aug. 17, 2010, Wikipedia®, Wikimedia Foundation, Inc (Retrieved on Sep. 9, 2010, at URL: http://en.wikipedia.org/wiki/Standardized_test). [23] Wikipedia, the free encyclopedia, “SMS language,” Aug. 27, 2010, Wikipedia®, Wikimedia Foundation, Inc (Retrieved on Sep. 9, 2010, at URL: http://en.wikipedia.org/wiki/SMS_language). [24] N. Wingfield, “Time to leave the laptop behind,” The Wall Street Journal, Feb. 23, 2009, page R1 (Retrieved on Sep. 9, 2010, at: http://online.wsj.com/article/SB122477763884262815.html). AUTHOR M. M. Alabbadi is with the Computer Research Insti- tute (CRI) at King Abdulaziz City for Science and Tech- nology (KACST), P. O. Box 6086, Riyadh - 11442, Saudi Arabia (e-mail: alabbadi@kacst.edu.sa). His research in- terests include mobile learning, wireless sensors networks (WSN), ubiquitous computing, cryptography, and infor- mation security. This work was supported in part by Elite Network, a Saudi-based com- pany. Submitted, September 10, 2010. Published as resubmitted by the authors on September 27, 2010. 10 http://www.i-jim.org http://metrics.admob.com/wp-content/uploads/2009/08/AdMob-Mobile-Metrics-July-09.pdf� http://metrics.admob.com/wp-content/uploads/2009/08/AdMob-Mobile-Metrics-July-09.pdf� http://www.ambientinsight.com/Resources/Documents/AmbientInsight_Learning_Technology_Taxonomy.pdf� http://www.ambientinsight.com/Resources/Documents/AmbientInsight_Learning_Technology_Taxonomy.pdf� http://www.pewinternet.org/~/media//Files/Reports/2008/PIP_FutureInternet3.pdf.pdf� http://www.pewinternet.org/~/media//Files/Reports/2008/PIP_FutureInternet3.pdf.pdf� http://www.time.com/time/nation/article/0,8599,1947019,00.html� http://knowledgetree.flexiblelearning.net.au/edition06/download/Geddes.pdf� http://knowledgetree.flexiblelearning.net.au/edition06/download/Geddes.pdf� http://www03.ibm.com/ibm/history/exhibits/specialprod1/specialprod1_9.html� http://www03.ibm.com/ibm/history/exhibits/specialprod1/specialprod1_9.html� http://www.nmc.org/pdf/2009-Horizon-Report.pdf� http://www.mlearn.org.za/CD/papers/keegan1.pdf� http://ftp.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-61/paper2.pdf� http://ftp.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-61/paper2.pdf� http://cit.mak.ac.ug/iccir/downloads/SREC_07/Paul Birevu Muyinda , Ezra Mugisa , Kathy Lynch_07.pdf� http://cit.mak.ac.ug/iccir/downloads/SREC_07/Paul Birevu Muyinda , Ezra Mugisa , Kathy Lynch_07.pdf� http://cit.mak.ac.ug/iccir/downloads/SREC_07/Paul Birevu Muyinda , Ezra Mugisa , Kathy Lynch_07.pdf� http://elearning.typepad.com/thelearnedman/mobile_learning/reports/futurelab_review_11.pdf� http://elearning.typepad.com/thelearnedman/mobile_learning/reports/futurelab_review_11.pdf� http://www.futurelab.org.uk/resources/documents/lit_reviews//Mobile_Review.pdf� http://www.futurelab.org.uk/resources/documents/lit_reviews//Mobile_Review.pdf� http://www.mlearn.org.za/CD/papers/Sharples Theory of Mobile.pdf� http://www.mlearn.org.za/CD/papers/Sharples Theory of Mobile.pdf� http://en.wikipedia.org/wiki/Standardized_test� http://en.wikipedia.org/wiki/SMS_language� http://online.wsj.com/article/SB122477763884262815.html� mailto:alabbadi@kacst.edu.sa� MOBIQIYAS: A MOBILE LEARNING STANDARDIZED TEST PREPARATION FOR SAUDI ARABIAN STUDENTS APPENDIX: SNAPSHOTS OF SOME OF THE QUESTIONS OF MOBIQIYAS FOR THE GCT TEST OF QIYAS iJIM – Volume 4, Issue 4, October 2010 11