Microsoft Word - cheung.doc Australasian Journal of Educational Technology 2009, 25(2), 153-183 A review of research methodologies used in studies on mobile handheld devices in K-12 and higher education settings Wing Sum Cheung and Khe Foon Hew Nanyang Technological University Mobile handheld devices are increasingly being used in education. In this paper, we undertook a review of empirical based articles to summarise the current research regarding the use of mobile handheld devices (personal digital assistants/PDAs, palmtops, and mobile phones) in K-12 and higher education settings. This review was guided by the following four questions: (a) How are mobile handheld devices such as PDAs, palmtops, and mobile phones used by students and teachers? (b) What types of research methods have been applied using such devices? (c) What data collection methods are used in the research? and (d) What research topics have been conducted on these handheld devices in education settings, as well as their related findings? We summarise and discuss some major findings from the research, as well as several limitations of previous empirical studies. We conclude by providing some recommendations for future research related to mobile handheld devices in education settings. Introduction In the last few years, mobile handheld devices have emerged as a tool for teachers and students to use in K-12 and higher education settings. In this article, we adapted Becta’s definition (cited in Perry, 2003) to refer to mobile handheld devices as any small machines that can be carried easily in one’s palm and provide computing, as well as information storage and retrieval capabilities. Mobile handheld devices differ from other mobile tools such as laptops because the latter, although portable, are typically not small and light enough to fit in one’s palm. Figure 1, adapted from Seppälä and Alamäki (2003), provides a pictorial illustration of the relationship between mobile handheld devices with laptops and desktop personal computers. One of the promises that mobile handheld devices hold is that of a one device to student ratio (Mifsud, 2004). Such a ratio enables a change from the occasional and supplemental use associated with computer laboratories, to more frequent and integral use of portable computer technology in teaching and learning (Roschelle, 2003; Tinker & Krajcik, 2001). Some researchers (e.g. Soloway et al., 2001) believe that such devices have the potential to revolutionise learning, allowing students to undertake learning activities wherever they happen to be. Furthermore, because mobile handheld devices are personal and portable, they may incite in learners a sense of personal ownership over learning tasks and the technologies used to support learning (Hennessy, 2000). The purpose of this article is to review the empirical literature pertaining to the use of mobile handheld devices in K-12 and higher education settings. In this article, we limit our review of handhelds to devices such as personal digital assistants (PDAs), 154 Australasian Journal of Educational Technology, 2009, 25(2) palmtops, and mobile phones. PDAs are shirt pocket sized devices equipped with computer capabilities (Baumgart, 2005; Churchill & Churchill, 2007). PDAs typically come with a touch sensitive screen, a pen/stylus input interface, customisable application buttons, and a multiway (button or mini joystick) navigator to browse information on the screen (Baumgart, 2005; Corbeil & Valdes-Corbeil, 2007). Nowadays, PDAs are generally equipped with a suite of personal information management software (e.g. calendar, notepad, address book), and can connect to desktop computers and wireless local area networks using infrared Bluetooth, or Wi-Fi communication technology (Baumgart, 2005). Palmtops are very similar to PDAs and often they are referred to PDAs by name. The main difference, however, is that palmtops typically have a built in keyboard compared to PDAs that need a stylus for data entry. Figure 1: Relationship between mobile handheld devices, laptops and desktop personal computers (adapted from Seppälä & Alamäki, 2003) Mobile phones started as voice communication tools; however, they rapidly become devices for text and image exchange, and recent models include built in cameras (Oliver & Goerke, 2007). The more sophisticated phones (sometimes known as smart phones) (Attewell, 2005) combine telephone capability with a PDA, camera, video, mass storage, MP3 player, Internet access, and networking features in one compact system (Corbeil & Valdes-Corbeil, 2007). In addition to email, these devices offer instant messaging (Reardon, 2007). Examples of smart phones include iPhone, Sony Ericsson P800/P900, and HP iPAQ hw6915. Our review of handheld devices was specifically guided by the following questions: a. How are mobile handheld devices such as PDAs, palmtops, and mobile phones used by students and teachers? b. What types of research methods have been applied using these handheld devices? c. What data collection methods are used in the research? d. What research topics have been conducted on these mobile handheld devices in education settings, as well as their related findings? Mobile Wireless Standard laptop Laptop with wireless access Desktop PC with wireless access Standard PDAs PDAs with wireless access, palmtops, mobile phones Handheld Cheung and Hew 155 Significance of this review This review of research not only helps researchers and educators identify the contemporary research topics, research methodologies, and usage of mobile handheld devices in K-12 and higher education settings, but also suggests directions for future research as well as some guidelines for the nature of that research. Method Searching and selection procedures The search for relevant literature was completed in two stages. First, we examined peer reviewed articles that we found in electronic databases using keyword searches including mobile learning, wireless learning, and handheld devices. We used Academic Search Premier, Business Source Premier, Communication and Mass Media Complete, ERIC, Library, Information Science and Technology Abstracts, and PsycARTICLES. In the second stage, we used the “snowball” method by searching for journal articles, as well as articles presented in peer reviewed conferences that are cited in some of the articles that we had read. Altogether, as at 28 December, 2008, we read 136 articles and deleted 92. The 92 articles were discarded because they were opinion papers, conceptual articles, non-empirical descriptions of program implementations, and literature reviews. The Appendix includes the remaining articles (n=44) which we included in our review of research. The 44 articles are listed according to the authors, year of publication, purpose, research method, data collection method, and context. Data analysis The basic unit of analysis was the individual empirical article. To answer the first research question, “How are mobile handheld devices such as PDAs, palmtops, and mobile phones used by students and teachers?”, we used Churchill and Churchill’s (2007) framework to guide our initial analysis and coding. This framework originally explicates a set of five ways in which PDAs may be used, namely as: (a) multimedia access tools, (b) communication tools, (c) capture tools, (d) representational tools, and (e) analytical tools. (See the Results section for a fuller description.) Although Churchill and Churchill’s framework was used a priori, we did not forcefully impose any of the coding categories onto our data corpus. During the course of our analysis, we also allowed for new categories (if any) to emerge inductively. To address the second, third, and fourth research questions, “What types of research methods have been applied using these handheld devices?”, “What data collection methods are used in the research?”, and “What research topics have been conducted on these mobile handheld devices?”, we employed the constant-comparative or grounded approach espoused by Lincoln and Guba (1985). This approach is similar to the iterative pattern coding method (Miles & Huberman, 1994). Specifically, the various categories of research methods, data collection methods, as well as research topics were not predetermined prior to our analysis but emerged inductively and were continually refined through our interaction with the data. 156 Australasian Journal of Educational Technology, 2009, 25(2) Results Uses of mobile handheld devices At the conclusion of our analysis, we had seven major categories of the uses of handheld devices such as PDAs, palmtops, and mobile phones in education. The seven categories include: (a) multimedia access tool, (b) communication tool, (c) capture tool, (d) representational tool, (e) analytical tool, (f) assessment tool, and (g) task managing tool. The first five categories originated from Churchill and Churchill’s (2007) framework, while the remaining two (i.e., assessment tool, and task managing tool) emerged inductively from the data via the constant-comparative approach (Lincoln & Guba, 1985). Multimedia access tools This refers to employing the PDAs, palmtops, or mobile phones as tools for accessing multimedia resources such as e-books, databases, web pages, PowerPoint presentations, audio files and video clips (Churchill & Churchill, 2007). For example, students in Maniar’s (2007) study watched an educational five minute video clip on how to measure blood pressure on a Nokia 6600 mobile phone, a Motorola E1000 phone, or a Compaq iPAQ H3800 PDA. Communication tools This refers to employing the handheld devices to communicate information from one person to others (Churchill & Churchill, 2007). Such communication may be established synchronously and asynchronously over mobile telephony or email. An example of such usage can be found in Seppälä and Alamäki’s (2003) study in which student teachers and their supervisors used a Nokia Communicator 9210 to communicate with one another (e.g., using short message service, SMS) during a teaching practicum. Capture tools This refers to utilising the devices to capture various data and media (Churchill & Churchill, 2007; Cochrane, 2008) such as video, still pictures, audio (e.g. interviews), or certain specific data such as water pollution indices via specially designed probes and sensors attached to the device. For example, students in Burke, Colter, Little and Riehl’s (2005) study utilised PDAs and temperature probes to gather on site data from grocery stores and restaurants. Students (e.g. pre-service teachers) might also use the inbuilt cameras available in mobile phones to capture digital pictures of their classrooms or lessons (Seppälä & Alamäki, 2003), use mobile phones as a video recording device to capture episodes of their lessons (Ferry, 2008), or record audio narrations using iPods with attached microphones (Olney, Herrington, & Verenikina, 2008). Representational tools PDAs, palmtops, or mobile phones may also be used by students to create representations that demonstrate or showcase their thinking, ideas, experiences and knowledge (Churchill & Churchill, 2007). Students in Dieterle and Dede’s (2006) study, for example, used PiCoMap, a concept mapping software to create maps with at least four nodes and four connections. The maps were subsequently beamed to other classmates; after which the students discussed similarities and differences among the various maps. Students in Polishook’s (2005) study used software such as NotePad, Cheung and Hew 157 BeatPad to compose music with their PDAs and explored how the PDAs might allow for artistic representation and expression that could not be duplicated with other tools. Analytical tools This refers to employing the devices to manipulate certain data or variables such as graphic calculators (Churchill & Churchill, 2007). An example of this could be found in Hennessy (2000), where 48 students aged 13-14 years used graphing facilities to plot temperature data which they had earlier captured using thermometers. Assessment tools This refers to the employment of handhelds for students to answer examination questions, tests, or quizzes. For example, Ganger and Jackson (2003) implemented the use of PDA based medical school exams in a wireless environment and reported the results of a student satisfaction survey of using such a tool. In another example, Fujimura and Doi (2006) implemented the use of mobile phones with Internet capability to assess students’ degree of comprehension of course content. Other examples of using handhelds as assessment tools can be found in Segall, Doolen and Porter (2005), Treadwell (2006), and Triantafillou, Georgiadou and Economides (2008). Besides using handhelds to conduct summative assessment (e.g. school exams), students may also utilise PDAs or mobile phones for the purpose of formative assessment. Specifically, students may use handhelds as a student response system tool to send responses to an instructor’s questions (usually in multiple choice format) in class (Chen, Myers, & Yaron, 2000; Jackson, Ganger, Bridge, & Ginsburg, 2005; Miyata & Kozuki, 2007). In such a system, students’ inputs are typically aggregated and displayed for the instructor and students. Aggregated responses can inform both teachers and students the overall distribution of the students’ responses in a classroom in real time (Fies & Marshall, 2006). An example of using handhelds as student response systems can be found in Jackson et al.’s (2005) study of wireless handheld computers in an undergraduate medical school. A series of instructor generated multiple choice questions were published on a portal for the students to access via PDAs during lecture. Students then responded to the questions in one to two minutes. The instructor and the class viewed the responses in real time allowing the instructor to modify the lecture content on the fly based on subject matter deficiencies shown by the students’ responses in the large group setting (Jackson et al., 2005), in order to meet students’ real and immediate learning needs (Beatty, 2004). Task management tools This refers to utilising the devices as personal information managers which store and organise a user’s address book, contact information, calendar, task lists (i.e. 'to-do-list'), documenting or recording student grades, attendance rates, or submission of homework (e.g. Corlett, Sharples, Bull & Chan, 2005; Dieterle & Dede, 2006; Franklin & Sexton, 2006; Jackson et al., 2005; Johnson & Wilkes, 2004; Olney & Lefoe, 2007; Sharples, Corlett, Bull, Chan & Rudman, 2005; Tyler-Wood, Rademacher, Dunn & Whitworth, 2007; Yamamoto & Akahori, 2006). Further analysis suggested that the three most frequent uses of the handhelds centered on utilising the devices as communication (21.8%), multimedia access (20.5%), and task management (17.9%) tools. These were followed by the use of handhelds as assessment (14.1%), capture tools (12.8%), representational (6.4%), and analytical tools (6.4%) (See Figure 2). 158 Australasian Journal of Educational Technology, 2009, 25(2) 20.5% 21.8% 12.8% 6.4% 6.4% 14.1% 17.9% 0% 5% 10% 15% 20% 25% Multimedia access Commun- ication Capture Represen- tational Analytical Assessment Task man- agement Percent Figure 2: Uses of mobile handheld devices Types of research methods In this section, we summarise the various research methods used in the previous studies we reviewed. A total of eight types of research methods were found: descriptive research, true experiment, experiment (weak), quasi-experiment, ex-post facto, single-subject, design-based research, and mixed method. Descriptive research Typically naturalistic and depicts conditions as they exist in a particular setting (Ross & Morrison, 1997). According to Knupfer and McLellan (1996), descriptive research is mainly concerned with what is type of questions that describe events focusing on a particular issue or phenomenon. True experiment Used to test hypotheses concerning causation (Ross & Morrison, 1997); for example, using PDAs leads to better student learning. Typically, in a true experimental design, there exist two groups: a treatment group and a control group. In addition, participants are randomly assigned to either group (Fraenken & Wallen, 2006). Experiment (weak) A typical weak experimental method is the one-group pretest-posttest design, in which a single group is measured both before and after a particular treatment (Fraenkel & Wallen, 2006). There is no comparison or control group. Quasi-experiment Similar to true experiment in that there exist two groups (i.e. a treatment group and a control group) but without the use of random assignment of participants to the groups (Fraeken & Wallen, 2006). Cheung and Hew 159 Ex-post facto Intact groups are used, however, the treatment is not manipulated (Ross & Morrison, 2004); the treatment has already occurred. Single subject design In this research method, data are collected and analysed for only one subject at a time; and is most commonly used in special education to examine the changes in an individual’s behaviour after exposure to an intervention or treatment (Fraeken & Wallen, 2006). Design based research This type of research typically involves both the design of certain forms of educational interventions based on a particular theoretical framework and systematically studying these forms in context, in order to better understand the various issues that target domain specific learning processes (Cobb, Confrey, diSessa, Lehrer & Schauble, 2003). Design-based research usually entails a continuous cycle of design, enactment, analysis, and redesign (Collins, 1992). Mixed method This specific type of research method includes studies that have more than one type of research method reported. For example, the first research method might be true experiment, while the second method was descriptive research (e.g. Thornton & Houser, 2005). Results indicate that on the whole, descriptive research was the most common type of research method (65.9%), followed by experiment (weak) (11.4%), mixed-method (6.8%), quasi-experiment (4.5%), true experiment (4.5%), ex-post facto (2.3%), single- subject design (2.3%), and design-based research (2.3%). Figure 3 illustrates the percent of the various types of research methods. 65.9% 11.4% 4.5% 4.5% 2.3% 2.3% 2.3% 6.8% 0% 10% 20% 30% 40% 50% 60% 70% 80% Descr res Exp (weak) True exp Quasi-exp Ex-post facto Single- subject Design- based res Mixed method Percent Figure 3: Types of research methods 160 Australasian Journal of Educational Technology, 2009, 25(2) Data collection methods The data collection methods that were found in our review of past empirical studies included the following five methods: test or quiz, questionnaire, interview or focus group discussion, observation, and content analysis. Test or quiz This data collection method may include pretest and posttest. Pretest is an instrument used to gather participants’ baseline performance data prior to an intervention or treatment of some sort. An example of pretest data is participants’ English vocabulary test or quiz scores before studying materials via mobile phone email. Posttest, on the other hand, is an instrument used to gather participants’ performance data after the conclusion of an intervention or treatment of some sort. An example of posttest data is participants’ English vocabulary test or quiz scores after studying materials via mobile phone email. Questionnaire A Likert-type scale of items used to collect data on participants’ satisfaction, or attitudes about a specific issue; for example students’ satisfaction with using mobile phones in learning English vocabulary. Interview or focus group We refer to interview as a verbal exchange or conversation between the researcher and an individual participant done either face to face or through the telephone. On the other hand, in a focus group interview, the researcher questions several individuals in small groups simultaneously (Fontana & Frey, 2000). Observation A data collection method in which the researcher directly watched participants in natural contexts or in contexts that are contrived to be realistic in order to get an indication of their behaviours or activities (Knupfer & McLellan, 1996). Content analysis A data collection method used by researchers to study participant behaviour or activity indirectly by gathering and examining the written contents of a communication (e.g. project plans, reflection logs, journals, emails, student worksheets, time logs, or text messages), usually through a process of comparison, and categorisation (Fraenken & Wallen, 2006; Schwandt, 1997). Results showed that 31.4% of all data collection methods used in previous studies were questionnaire, 22.5% were test or quiz items, 20.6% were content analysis, 18.6% were interview or focus group, and 6.9% were observation (see Figure 4). Research topics and findings A total of four main research topics were found: usage profile, viability as an assessment tool, learning outcomes, and attitudes. Usage profile This topic of research typically examines: types of handhelds owned, frequency of use of devices, as well as the purposes for using them. For example, Thornton and Houser (2005) studied 333 Japanese students regarding their usage of mobile phones. Students’ Cheung and Hew 161 22.5% 31.4% 18.6% 6.9% 20.6% 0% 5% 10% 15% 20% 25% 30% 35% Test/quiz Questionnaire Interview/ focus group Observation Content analysis Percent Figure 4: Data collection methods ages ranged from 18 to 21, and disciplines of study included English as a foreign language, modern culture, computers, design, and home economics. Overall results showed that email was the most utilised mobile phone feature with an average of almost 200 email messages per week. Students (83%) mainly used mobile email for chatting with friends, 66% used it to ask other students about classes, and 44% used it for studying. Only 20% of students had used a PDA. Other studies (e.g., van ‘T Hooft, Diaz, & Swan, 2004) examined 217 grades 1-12 students’ usage of PDAs, and found that 29% used them one to two times a week, 37% reported three to four times a week, and 32% reported using them almost every day. Kennedy, Judd, Churchward, Gray & Krause (2008) reported that almost 80% of 2,120 students used their mobile phones to SMS people daily, and 57.2% of students used them to take pictures on a daily or weekly basis. The researchers found that some functions of mobile phones had yet to enjoy a wider usage. For example, a vast majority of students have not used their mobile phones to access information on the web (67.8%) or to send and receive email messages (75.8%). Churchill and Churchill (2007) found that the purposes of using PDAs included accessing multimedia resources, communicating between students-students and students-teachers, capturing data, representing students’ ideas or knowledge as in concept maps, and analysing and manipulating data. Viability as an assessment tool This topic of research examines the usability of mobile handheld devices such as PDAs for assessments such as school based examinations, tests or quizzes. In our review of the research, we found four studies that dealt explicitly with this research topic. Of these four, two studies compared the use of PDAs versus paper and pencil in terms of student test scores, efficiency, and student satisfaction (Segall et al., 2005; Treadwell, 2006), one examined only the satisfaction aspect (Ganger & Jackson, 2003), while one study investigated the design and development issues regarding the implementation of a computerised adaptive test (CAT) on PDAs (Triantafillou et al., 2008). 162 Australasian Journal of Educational Technology, 2009, 25(2) Overall, previous empirical results suggested that there was no statistically significant difference in terms of test scores between students who used the PDAs and students who used the paper based assessment method (Segall et al, 2005; Treadwell, 2006). Efficiency may be defined as the resources consumed in order to obtain a goal (Segall et al, 2005). Some of the efficiency measurements reported in previous studies included time to complete a quiz or test, and time to prepare a quiz. Results with regard to efficiency in terms of time to complete a quiz or test were not conclusive. Although Segall et al’s (2005) results suggested that although the PDA based quiz was more efficient to complete compared to the paper and pencil quiz, the results should be interpreted carefully because the two quizzes were dissimilar in the material covered. On the other hand, the results for efficiency in terms of time to prepare a quiz or test seemed to indicate that PDA based assessment took a lesser time to prepare compared to the paper and pencil based assessment (Treadwell, 2006). Results pertaining to student satisfaction were mixed. One study (Ganger & Jackson, 2003) indicated that students preferred paper and pencil assessment to PDA based assessment, because the latter did not permit flagging questions for later review; each question in the PDA was answered and submitted individually, hence students were unable to review or change their responses. Segall et al (2005) found no significant difference in student satisfaction between the PDA based and paper and pencil assessment methods. On the other hand, other studies (e.g. Treadwell, 2006; Triantafillou et al., 2008) suggested that students were satisfied with a PDA based assessment system. For example, students in Triantafillou et al’s (2008) study indicated that the test was clear and secure, and that the use of the PDA was very interesting and attractive. Learning outcomes This research topic investigates whether the use of mobile handheld devices can improve or enhance students’ learning. In our review of the research, we found 11 studies (Chen & Chung, 2007; Chen, Kao & Sheu, 2003; Hennessy, 2000; Kong & Li, 2007; Maniar, 2007; Moallem, Kermani & Chen, 2003; Schcolnik, Kol & Oren, 2007; Tan et al., 2007; Thornton & Houser, 2005; Zurita & Nussbaum, 2004) that dealt with this research topic. Of these 11 studies, three compared the performance of students who used PDAs or mobile phones with students who used other methods such as books, desktops, or paper. These studies were done primarily using quasi-experimental or true experimental research methods. For example, Chen et al (2003) examined whether elementary students’ performance (pretest and posttest) of bird knowledge benefited from using PDAs. Students in the PDA group used the handheld devices to observe a static image of the bird, search for information about birds, and answer multiple questions on features of birds. The control group, on the other hand, was given a two- cone telescope and a guidebook. This group identified bird features with the telescope, searched for information with guidebook, received questions orally, and provided answers on a paper worksheet. Results suggested that students using the PDA significantly outperformed students who were using guidebooks. The researchers posited that this was due to the students having to spend more mental effort for completing the task on their own using the PDA. However, what such mental effort entailed was not explained at all. In another comparison study, Thornton and Houser (2005) assessed students’ learning of English vocabulary via mobile phones compared to using the Web or paper based materials. The researchers of the study conducted two quasi-experiments. In the first Cheung and Hew 163 quasi-experiment, a group of students (n=13) studied English vocabulary lessons over a two week period. These lessons were emailed to the students’ mobile phones. Another group of students (n=13) were encouraged to study identical materials on an identical schedule, materials that the researchers had posted on a website. After two weeks, the two groups of students switched media for another two weeks. In the second quasi-experiment, two groups of students studied identical materials for two weeks. The researchers sent messages to one group’s (n=25) mobile phones, and encouraged the other group of students (n=43) to study identical messages that they had printed on paper and distributed to the students. Overall results indicated that students who received mobile e-mail English vocabulary lessons at timed intervals learned more compared to students who were urged to regularly study identical materials on paper or Web. The researchers posited that the push aspect of mobile phones promoted regular study, and this more frequent study led to better learning. Zurita and Nussbaum (2004) examined a learning environment supported by wireless handhelds (PDA-like devices) to teach Spanish words for first graders in Chile. Specifically, the first graders were randomly grouped in three, where each child had one Spanish syllable. The children had to find out how to combine the syllables to build words known to them. One environment (the control group) was implemented without any technological support, while the other environment (the experimental group) was with the support of wireless inter-connected handhelds. There was a significant difference between the two groups in terms of their posttest scores in word construction with the experimental group performing better than the control group. The control group had problems (e.g. children quarelling) when dealing with the syllable tokens. Also, there were little or no discussion over a word to construct since one child imposed his or her personal point of view. On the other hand, the handhelds offered an environment where there was no arguing over the dealing of syllables (each child has one handheld) and the cognitive effort was targeted to the construction of words. Also, the handhelds gave each child the possibility to decide how his or her syllable was used in relation to the rest of the group, and then whether the formed word was correct. Other studies (n=5), using weak experimental research methods, also attempted to determine whether the use of mobile handheld devices can enhance students’ learning, but without the use of a control group (Chen & Chung, 2007; Hennessy, 2000; Kong & Li, 2007; Moallem, Kermani & Chen, 2003; Tan et al., 2007). With various student populations (university undergraduates, fourth, seventh and eighth graders), and different subject matter contexts (English language, mathematics, education, environmental issues), and employing mainly pretest and posttest scores, it seems that the results of these studies overall suggest that using such handhelds can enhance student learning. One study (Levi & Kennedy, 2005) used students’ self reports such as questionnaire and focus group data to measure the effectiveness of learning Italian via mobile SMS. The study did not employ any control group. Participants indicated that utilising mobile SMS could help their learning of Italian because the SMS served as a useful trigger or prompt to help students reinforce what they had earlier learned in class. Finally, some researchers were interested in investigating how different screen sizes might affect students’ learning performance (Maniar, 2007; Schcolnik, Kol & Oren, 2007). For example, each student (n=16) in Maniar’s study watched an educational five-minute video on how to measure blood pressure on their allocated devices such as 164 Australasian Journal of Educational Technology, 2009, 25(2) - (a) small screen (Nokia 6600 mobile phone, screen size 42 mm diagonal), (b) medium screen (Motorola E1000 mobile phone, 58 mm), and (c) large screen (Compaq iPAQ H3800 PDA, 96 mm). Before watching the video, participants were asked four questions to assess their prior knowledge. After watching the video, they were asked the same four questions again. Maniar found that participants using the large and medium screen devices learnt significantly more than those using the small screen. No significant difference in the amount learnt between participants using the large screen and those using the medium screen. Overall, this suggests that screen sizes of approximately 58 mm (diagonal) and above can result in a significantly higher amount of information learnt compared to sizes of around 42 mm and below. Attitudes In addition to studying learning outcomes, researchers were also interested in outcomes in the affective domain, such as students’ perceptions of desired functions in a handheld device, as well as students’ satisfaction toward using PDAs and mobile phones. For example, in a survey involving 84 nurse students in their third year at the Department of Health and Behavioral Sciences, Berglund, Nilsson, Révay, Petersson and Nilsson (2007) found that the six most frequently mentioned functions that nurse students desired in a PDA included (a) calculator, (b) camera, (c) journal notes, (d) alarm function, (e) calendar, and (f) Internet/intranet/mail. With regard to students’ satisfaction, research overall seems to suggest that students like using the handhelds due to the following reasons: portability of the handhelds which made it very convenient for students to use waiting or traveling moments to do work using the devices, price advantage of the handhelds over laptops, ease of synchronisation of the handhelds with desktop computers by infrared (beaming), and instant-on capability of the devices (no waiting for an operating system to boot up) (e.g., Hennessy, 2000; Motiwalla, 2007; Perry, 2003; Seppälä & Alamäki, 2003; Van ‘T Hooft, Diaz & Swan, 2004; Waycott, Jones & Scanlon, 2005). On the other hand, some of the reported dislikes about using the handhelds include: unfamiliarity with the devices, limited battery life, limited memory, small screen size, instability of the devices due to hardware crash, difficulties in making inputs using the stylus or phone keypads, and limited Internet browsing due to a majority of websites not rendering in a usable format on small screens (e.g. Chen, Myers & Yaron, 2000; Burke et al., 2005; Corlett, Sharples, Bull & Chan, 2005; Franklin & Sexton, 2006; Johnson & Wilkes, 2004; Olney & Lefoe, 2007; Ramsden, 2005; Seppälä & Alamäki, 2003). Results showed that 49.1% of all research topics were about learners’ attitudes toward the use of mobile devices, 26.3% centered on usage profile, 17.5% were about learning outcome, and finally 7.0% focused on the viability of mobile handheld devices as an assessment tool (see Figure 5). Discussion and conclusion In this paper, we undertook a review of extant, empirical based research to summarise the current literature regarding the use of mobile handhelds devices such as PDAs, palmtops, and mobile phones, in K-12 and higher education settings. In this section, we summarise and discuss some major findings from the research, as well as several limitations of previous empirical studies. We conclude by providing some recommendations for future research related to mobile handheld devices in education. Cheung and Hew 165 49.1% 17.5% 26.3% 7.0% 0% 10% 20% 30% 40% 50% 60% Attitudes Learning outcome Usage profile Assessment tool Percent Figure 5: Research topics Several major findings can be summarised from the research. These include the following: 1. Mobile handheld devices are most commonly used by students and teachers as communication and multimedia access tools (i.e. accessing multimedia resources such as e-books, databases, web pages, PowerPoint presentations). The widespread use of mobile devices as a communication tool is intuitively reasonable since these devices were originally designed and created for people to exchange information with one another. The dominant use of mobile devices (e.g. PDAs) as multimedia access tools suggests that the current technology functions primarily as replacement. According to Hughes (2005), technology as replacement involves technology serving as a different means to the same instructional or learning goal. For example, students could read an e-book on a PDA. This activity replaces the reading of a paper based book with the unchanged learning goal of student gaining information. Or teachers asking their students to access web pages and PowerPoint presentations on the go using PDAs, instead of being in a physical computer lab using a desktop computer or classroom with the unchanged instructional goal for students to gain information. A possible driving force for the wide use of PDAs as replacement technology may be due to the convenience and portability of the handheld devices. Technology as replacement may be contrasted with technology functioning as transformation (Hughes, 2005). The latter has the potential to reorganise students’ cognitive processes and problem solving activities such as using graphing tools for exploratory data analysis, data organisation, and testing hypotheses related to the data (Pea, 1985). Another plausible reason for the current wide use of PDAs as replacement technology is that students and teachers are new to using such devices in education; hence they are still exploring them. Technology integration stage theorists would view such users as being located in the familiarisation stage (Hokanson & Hooper, 2004). 2. Descriptive research is the most dominant type of research method employed by previous empirical studies. This result supports Knupfer and McLellan’s (1996) 166 Australasian Journal of Educational Technology, 2009, 25(2) prognosis that there would be more studies employing descriptive research as their research method in recent years to explain educational events or issues. Another probable reason for conducting descriptive research is that it is better placed to answer complex educational questions situated in diverse contexts. 3. The most common data collection method used is the questionnaire. One possible reason for this might be the ability of the questionnaire to collect data from a large pool of participants relatively easily compared to other methods such as observation, or content analysis. 4. There is no significant difference in terms of test scores achieved by students in the PDA and paper and pencil based assessment methods. Results, however, seem to suggest that the PDA based assessment method is more efficient than paper and pencil method in terms of time to prepare the quiz or test; but is inconclusive whether the PDA based method is more efficient in terms of time to complete the test. Results pertaining to student satisfaction of using PDA as an assessment tool are mixed. 5. The majority of results appear to suggest that students’ learning is enhanced through the use of mobile handheld devices. This finding, however, should be viewed with caution because of the lack of rigorous research designs. We will describe this issue in greater detail in a later section of this article, “Limitations of previous studies.” Results also suggested that previous studies that examined students’ learning appeared to focus on two types of Gagne’s (1985) learning outcomes – verbal information, and intellectual skills. For example, with regard to the former, five of the ten studies dealt with language learning, in particular the learning of English vocabulary (e.g. Chen & Chung, 2007; Thornton & Houser, 2005). The remaining five studies appeared to deal with an intellectual skills outcome such as discriminating different types of birds (e.g. Chen et al., 2003). 6. In our review of previous research, we found that a majority of the studies tended to place greater emphasis on the features of the mobile devices and procedures for using them, rather than on the theoretical rationale or justification for using them. The use of theoretical foundations will help inform the pedagogy of using them – a key success factor to the successful use of mobile devices to enhance learning as claimed by Duncan-Howell and Lee (2007). We recommend that researchers invoke appropriate theoretical foundations to inform future research. Some of the theoretical foundations relevant to learning and teaching in a mobile learning environment might include activity theory (Smørdal & Gregory, 2003; Waycott et al., 2005) and conversation theory (Kukulska-Hulme & Traxler, 2005; Pask, 1975). For example, Waycott et al. (2005) described the application of activity theory to analyse how online students adapted and appropriated PDAs for their learning activities, what benefits the mobile devices enabled, and what problems students encountered. Conversation theory describes learning in terms of communication with teachers, as well as with other students (Laurillard, 2002; Sharples, Corlett & Westmancott, 2002). It is through mutual conversation that one comes to a shared understanding of an issue or topic (Sharples et al., 2002). Motiwalla (2007) addressed the application of conversational theory to mobile phones by describing how the push mechanism of SMS can help support conversations between students on a particular topic in a mobile learning environment. Cheung and Hew 167 7. The use of Everett Rogers’ diffusion of innovation theory can also provide one with a possible framework to examine the uptake of mobile devices in teaching and learning. Specifically, Rogers (2003) described an innovation as “…an idea, practice, or object that is perceived as new by an individual or other unit of innovation” (p. 12). Although many would argue that mobile devices such as mobile phones and PDAs are not really innovative objects in the 21st century era, Rogers asserted that an innovation need not be a brand new thing, in contrast to what its name implies, but rather is perceived as new by the user. We believe that this is certainly the case with the use of mobile devices in K-12 and higher education contexts. Using Rogers’ diffusion theory to reflect on previous research studies done on mobile devices has enabled us to make the following observations. First, we noted that for many studies, the uptake of mobile devices was mainly limited to the level of knowledge awareness, possibly due to the short duration of time in which the studies were conducted. A majority of the studies (56%) were limited in their duration, ranging from a few hours to one semester. Short-term studies do not allow users the opportunity to fully adopt the innovation because students and teachers are still new to using such devices in education; hence they are still exploring them. Second, we noted that one aspect of the innovation – complexity, which is the degree to which an innovation is perceived as difficult to use, can hinder the uptake of mobile devices in K-12 and higher education settings. While many users appreciated the portability of the devices, the results of our review suggested that users were also hindered by issues such as limited battery life, limited memory space, small screen size, difficulties with inputs using the stylus or phone keypads, and limited Internet browsing due to a majority of websites not rendering in a usable format on small screens. However, issue such as difficulties in inputing data using phone keypads, or small screens may perhaps be an irresolvable dilemma because any attempt to increase screen size or allow for a greater ease of data entry (e.g. via a keyboard), may render the devices bulkier and therefore less portable. 8. Some authors have suggested that the cost of using mobile handheld devices (e.g. price of the device, cost of mobile service) is a hindrance to introducing the devices into teaching and learning (Churchill & Churchill, 2007; Kukulska-Hulme & Traxler, 2005; Lindquist et al., 2007; Litchfield et al., 2007; McCracken, Withers & Fee, 2007; Perry, 2003). However, other authors seemed to disagree (Allan, 2006; Moallem et al., 2003; Motiwalla, 2007; Thornton & Houser, 2005; van‘T Hooft et al., 2004; Zurita & Nussbaum, 2004). For example, Allan 2006 argued that although the cost of a mobile device (e.g., a PDA) varies according to functionality, size of memory, and accessory features, it is usually less expensive than desktop and laptop computers. As a result, schools and universities could afford to buy and use PDAs in teaching and learning more than they could with laptops. Motiwalla (2007) claimed that students could have more access to data services in the future due to the newer digital phones having low-cost access to fixed-rate data services from 3G to WiFi network services. Although it is important to examine the cost of usage (e.g., price of phone, cost of SMS messages) per se, we believe that it may be more crucial to investigate the cost effectiveness of using a mobile handheld device. Cost effectiveness compares the costs and actual outcomes of a particular course of action in order to determine whether it is indeed worth doing it. Universities would be interested to know if the money used to introduce devices (e.g. PDAs, mobile phones) could affect certain outcomes, for example whether student enrollment is positively affected by use of these devices in completely online courses. Brown 168 Australasian Journal of Educational Technology, 2009, 25(2) (2005) suggested that the use of handhelds can make “learning even more widely available and accessible than we are used to in existing e-learning environments” (p. 299). Indeed, if more students choose to enroll in online courses that utilise such devices, it would help justify the money that schools must spend for using them. Limitations of previous empirical studies There are several methodological limitations concerning previous empirical studies. First, a majority of studies used a weak experimental method that utilised a one group pretest and posttest design to examine student learning outcomes due to use of mobile handheld devices use. According to Tallent-Runnels et al. (2006), without a control group, differences found between pretest and posttest scores might not necessarily be attributable to the use of mobile handheld devices such as a PDA or mobile phone. Adding a control group and random assignment of participants to the research design could strengthen a study considerably (Tallent-Runnels et al., 2006). While this may be true, we are hesitant to advocate media comparison studies per se (e.g., one group with PDA versus one group without PDA). This is because historically researchers in instructional technology have voiced concerns about comparing learning outcomes between different media due to the presence of many potential confounding variables which makes the task of pinpointing a specific factor or factors that may account for the learning difference difficult, if not impossible (Clark, 1983). Instead, since different media possess different attributes (Kozma, 1994), future research studies might focus on investigating where and how to best use mobile devices as an instructional medium. In recent years, researchers have advocated the use of design based research as a tool to study the use of educational technologies in authentic complex settings (e.g. Dede, 2005; Collins, Joseph & Bielaczyz, 2004). Collins et al (2004) argued that, “They [design based methods] fill a niche in the array of experimental methods that is needed to improve educational practices.” (p. 21). Future research studies could employ the design based research approach to researching the use of mobile devices in K-12 and higher education contexts. The second limitation is that a great majority of the previous studies did not report effect sizes in the results or discussion section (the exception being Tan et al’s 2007 study). The APA Task Force stressed that researchers should provide some effect size estimate such as Cohen’s d when reporting a p value since reporting and interpreting effect sizes is essential to good research (Wilkinson & APA Task Force on Statistical Inference, 1999). Kotrlik and Williams (2003) argued that reporting effect size allows a researcher to judge the magnitude of the differences present between groups, thus increasing the capability of the researcher to judge the practical significance of the results derived. The third limitation is that almost half (46%) of the descriptive research studies based their findings primarily on participants’ self-reported data, such as interviews and questionnaire surveys. A general problem of studies based on self-reported data is that participants usually have correct notions about socially desirable answers, which can be referred to as the tendency to provide answers that cause the respondent to look good (Hakkarainen et al., 2001; Rosenfeld, Booth-Kewley, Edwards & Thomas, 1996). Social desirability responding has long been viewed as a potential source of error variance in self-report measures (Hancock & Flowers, 2001). In addition, studies that employed observations as a means to collect data did not report any inter-observer or intra-observer agreement reliability. Knupfer and McLellan (1996) argued that because human observers may have biasing expectations, and their recording methods may Cheung and Hew 169 change over time due to fatigue or practice, it is important that an assessment of both inter-observer and intra-observer reliability be conducted for observational research. The fourth limitation is that a majority of the studies (56%) were limited in their duration, ranging from as short as a few hours to one semester. Short-term studies cannot fully address some issues that may be critical in helping us understand how the use of mobile handheld devices might evolve over time. For example, studies that are short-term may suffer from a novelty effect, where it is possible that students and teachers are more likely to use the devices because the devices are new to them compared to participants who have used them for a longer period of time. This may introduce a significant bias with respect to the obtained results. Such a possibility, has indeed, been raised in one of the studies (Allan, 2006) who found that seventh graders had a preference for PDAs, while the ninth graders were divided in their responses. The difference between the ninth and seventh graders could be attributed to the novelty factor because the seventh graders only used the PDAs for less than one year, while the ninth graders, who had used the devices for almost three years, voiced their frustrations with the device's small screen size. Suggestions for future research We suggest that future research should provide a rich, thick description of the methodology, including the length of the study (8 of the 39 studies did not report the study duration), inter-observer and intra-observer agreement reliability, and effect sizes, so that findings can be adequately interpreted. Second, future studies should be longitudinal in nature, perhaps more than one year. Doing longitudinal studies not only provides researchers the opportunity to examine whether students’ and teachers’ perceptions of mobile handheld devices undergo change, but also to examine if the reported positive impact on students’ learning outcomes hold over time. In addition, longitudinal studies can help reveal any detrimental effects of using the mobile handheld devices over a long period of time, which hitherto has not been investigated at all. Future studies might also examine the use of mobile handheld devices in relation to other types of learning outcomes besides verbal information and intellectual skills domains. For example, future studies should explore cognitive processes through problem solving, investigations and other inquiry based approaches using handheld devices. Finally, we urge future researchers to examine the influence of cultural and geographical contexts on use of mobile handheld devices. So far in our review, a majority of the studies involved participants from countries in North America and Europe, compared to other countries. There is a continual need to study participants in other countries in order to better understand how different cultural contexts might influence the use of mobile handheld devices differently. References Allan, C., Carbonaro, M. & Buck, G. (2006). A survey of personal digital assistants (PDAs) use in a middle school environment: Perceptions of teachers, parents and students. Meridian Middle School Computer Technologies Journal, 2(9). [viewed 22 Nov 2007, verified 5 Apr 2009] http://www.ncsu.edu/meridian/sum2006/PDAS/ Attewell, J. (2005). Mobile technologies and learning: A technology update and m-learning project summary. London: Learning and Skills Development Agency. [verified 5 Apr 2009] http://www.lsneducation.org.uk/user/order.aspx?code=041923&src=XOWEB 170 Australasian Journal of Educational Technology, 2009, 25(2) Baumgart, D. C. (2005). Personal digital assistants in health care: Experienced clinicians in the palm of your hand? The Lancet, 366, 1210-1222. Beatty, I. (2004). Transforming student learning with classroom communication systems. Educause Center for Applied Research Bulletin, 3, 1-13. [verified 5 Apr 2009] http://educause.edu/ir/library/pdf/ERB0403.pdf Berglund, M., Nilsson, C., Révay, P., Petersson, G. & Nilsson, G. (2007). Nurses’ and nurse students’ demands of functions and usability in a PDA. International Journal of Medical Informatics, 76, 530-537. Brown, T. (2005). Towards a model for m-learning in Africa. International Journal on E-learning, 4(3), 299-315. Burke, M., Colter, S., Little, J. & Riehl, J. (2005). Utilizing wireless mobile devices to implement field-based collaborative learning strategies and activities at the University of Tennessee. In P. Kommers & G. Richards (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2005 (pp. 2198-2202). Chesapeake, VA: AACE. Chen, C-M. & Chung, C-J. (2007). Personalized mobile English vocabulary learning system based on item response theory and learning memory cycle. Computers & Education, 51(2), 624-645. Chen, F., Myers, B. & Yaron, D. (2000). Using handheld devices for tests in classes (Technical Report CMU-CS-00-152). Pittsburgh: Carnegie Mellon University, School of Computer Science. [verified 5 Apr 2009] http://www.cs.cmu.edu/~pebbles/papers/CMU-CS-00-152.pdf Chen, Y.S., Kao, T.C. & Sheu, J. P. (2003). A mobile learning system for scaffolding bird watching learning. Journal of Computer Assisted Learning, 19, 347-359. Churchill, D. & Churchill, N. (2007). Educational affordances of PDAs: A study of a teacher’s exploration of this technology. Computers & Education, 50, 1439-1450. [verified 5 Apr 2009] http://www.pgce.soton.ac.uk/ict/SecondaryICT/PDFs/Educational%20affordances%20 PDAs%20tescher%20exploration%20of%20technology%20mobile.pdf Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445-459. Cobb, P., Confrey, J., diSessa, A., Lehrer, R. & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9-13. Cochrane, T. (2008). Mobile Web 2.0: The new frontier. In Hello! Where are you in the landscape of educational technology? Proceedings ascilite Melbourne 2008. http://www.ascilite.org.au/conferences/melbourne08/procs/cochrane.pdf Collins, A. (1992). Toward a design science of education. In E. Scanlon & T. O’Shea (Eds.), New directions in educational technology. New York: Springer-Verlag. Collins, A., Joseph, D. & Bielaczyc, K. (2004). Deisgn research: Theoretical and methodological issues. Journal of the Learning Sciences, 13(1), 15-42. Corbeil, J. R. & Valdes-Corbeil, M. E. (2007). Are you ready for mobile learning? Educause Quarterly, 30(2), 51-58. http://www.educause.edu/ir/library/pdf/eqm0726.pdf Corlett, D., Sharples, M., Bull, S. & Chan, T. (2005). Evaluation of a mobile learning organizer for university students. Journal of Computer Assisted Learning, 21, 162-170. Dede, C. (2005). Why design-based research is both important and difficult. Educational Technology, 45(1), 5-8. Dieterle, E. & Dede, C. (2006). Straightforward and deep effects of wireless handheld devices for teaching and learning in university settings. Paper presented at the 2006 American Educational Research Association Conference, San Francisco: CA. [verified 5 Apr 2009] http://gseacademic.harvard.edu/~hdul/AERA_2006_Dieterle_Dede.pdf Cheung and Hew 171 Duncan-Howell, J. & Lee, K.T. (2007). M-learning: Finding a place for mobile technologies within tertiary educational settings. In ICT: Providing choices for learners and learning. Proceedings ascilite Singapore 2007. http://www.ascilite.org.au/conferences/singapore07/procs/duncan- howell.pdf Ferry, B. (2008). Using mobile phones to augment teacher learning in environmental education. In Hello! Where are you in the landscape of educational technology? Proceedings ascilite Melbourne 2008. http://www.ascilite.org.au/conferences/melbourne08/procs/ferry.pdf Fies, C. & Marshall, J. (2006). Classroom response systems: A review of the literature. Journal of Science Education and Technology, 15(1), 101-109. Fontana, A. & Frey, J. H. (2000). The interview: From structured questions to negotiated text. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 645-672). Thousand Oaks, CA: Sage Publications. Fraenkel, J. R. & Wallen, N. E. (2006). How to design and evaluate research in education (6th edition). McGraw-Hill, New York. Franklin, T. & Sexton, C. (2006). Faculty use of handhelds in higher education: Three years of study. In C. Crawford et al. (Eds.), Proceedings of Society for Information Technology and Teacher Education International Conference 2006 (pp. 2201-2206). Chesapeake, VA: AACE. Fujimura, N. & Doi, M. (2006). Collecting students' degree of comprehension with mobile phones. In Proceedings of the 34th Annual ACM SIGUCCS Conference on User Services (pp. 123- 127). Edmonton, Alberta: Canada. Gagne, R. M. (1985). The conditions of learning. (4th ed.). New York: Holt, Rinehart & Winston. Ganger, A. C. & Jackson, M. (2003). Wireless handheld computers in the preclinical undergraduate curriculum. Medical Education Online, 8(3). http://www.med-ed- online.org/pdf/t0000031.pdf Hakkarainen, K., Muukonen, H., Lipponen, L., Ilomaki, L., Rahikainen, M. & Lehtinen, E. (2001). Teachers’ information and communication technology skills and practices of using ICT. Journal of Technology and Teacher Education, 9(2), 181-197. Hancock, D. R. & Flowers, C. P. (2001). Comparing social desirability responding on World Wide Web and paper-administered surveys. Educational Technology Research and Development, 49(1), 5-13. Hennessy, S. (2000). Graphing investigations using portable (palmtop) technology. Journal of Computer Assisted Learning, 16, 243-258. Hokanson, B. & Hooper, S. (2004). Integrating technology in classrooms: We have met the enemy and he is us. Paper presented at the Annual Meeting of the Association for Educational Communications and Technology, Chicago: IL. [verified 5 Apr 2009] http://hokanson.cdes.umn.edu/publications/IntegratingAECT.pdf Hughes, J. (2005). The role of teacher knowledge and learning experiences in forming technology-integrated pedagogy. Journal of Technology and Teacher Education, 13(2), 277-302. Jackson, M., Ganger, A. C., Bridge, P. D. & Ginsburg, K. (2005). Wireless handheld computers in the undergraduate medical curriculum. Medical Education Online, 10(5). http://www.med- ed-online.org/pdf/t0000062.pdf Johnson, D. W. & Wilkes, F. (2004). Using handheld computers to support collaborative research and learning: A course experiment. Issues in Information Systems, 5(1), 145-151. http://iis- online.org/iis/2004_iis/PDFfiles/JohnsonWilkes.pdf Kennedy, G. E., Judd, T. S., Churchward, A., Gray, K. & Krause, K.-L. (2008). First year students' experiences with technology: Are they really digital natives? Australasian Journal of Educational Technology, 24(1), 108-122. http://www.ascilite.org.au/ajet/ajet24/kennedy.html 172 Australasian Journal of Educational Technology, 2009, 25(2) Kong, S. C. & Li, C. S. (2007). A study of using a cognitive tool in a mobile technology supported classroom. In T. Hirashima, U. Hoppe & S. S-C. Young (Eds.), Supporting learning flow through integrative technologies (pp. 455-462). Amsterdam, The Netherlands: IOS Press. Kotrlik, J. W. & Williams, H. A. (2003). The incorporation of effect size in information technology, learning, and performance research. Information Technology, Learning, and Performance Journal, 21(1), 1-7. [verified 5 Apr 2009] http://www.osra.org/itlpj/kotrlikwilliamsspring2003.pdf Kozma, R. B. (1994). Will media influence learning? Reframing the debate. Educational Technology Research and Development, 42(2), 7-19. Knupfer, N. N. & McLellan, H. (1996). Descriptive research methodologies. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (pp. 1196-1212). New York: Macmillan. Kukulska-Hulme, A. & Traxler, J. (2005). Mobile teaching and learing. In A. Kukulska-Hulme & J. Traxler (Eds.), Mobile learning: A handbook for educators and trainers (pp. 25-44). London and New York: Routledge. Laurillard, D. (2002). Rethinking university teaching: A conversational framework for the effective use of learning technology (2nd ed.). London: Routledge. Levy, M. & Kennedy, C. (2005). Learning Italian via mobile SMS. In A. Kukulska-Hulme & J. Traxler (Eds.), Mobile learning: A handbook for educators and trainers (pp. 76-83). New York, NY: Routledge, Taylor Francis Group. Lincoln, Y. & Guba, E. (1985). Naturalistic inquiry. Newbury Park, CA: Sage Publications. Lindquist, D., Denning, T., Kelly, M., Malani, R., Griswold, W. G. & Simon, B. (2007). Exploring the potential of mobile phones for active learning in the classroom. Proceedings of the 38th SIGCSE Technical Symposium on Computer Science Education, Vol. 39, No. 1, 384-388. [verified 5 Apr 2009] http://www-cse.ucsd.edu/~wgg/Abstracts/fp142-lindquist.pdf Litchfield, A., Dyson, L., Lawrence, E. & Zmijewska, A. (2007). Directions for m-learning research to enhance active learning. In ICT: Providing choices for learners and learning. Proceedings ascilite Singapore 2007. http://www.ascilite.org.au/conferences/singapore07/procs/litchfield.pdf Maniar, N. (2007). M-learning to teach university students. In C. Montgomerie & J. Seale (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2007 (pp. 881-887). Chesapeake, VA: AACE. McCracken, J., Withers, D. & Fee, J. (2007). The contexts and everyday uses of mobile technologies for first year university students: Implications for learning activity design. In C. Montgomerie & J. Seale (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2007 (pp. 2077-2080). Chesapeake, VA: AACE. Moallem, M., Kermani, H. & Chen, S. (2003). Integrating wireless computer technology into classroom instruction. In C. Crawford et al. (Eds.), Proceedings of Society for Information Technology and Teacher Education International Conference 2003 (pp. 1398-1401). Chesapeake, VA: AACE. Mifsud, L. (2004). Research in the use of handheld technologies in compulsory education: A review of literature. Paper presented at IRIS’27, Faljenberg, Sweden. Miles, M. B. & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: Sage Publications. Motiwalla, L. F. (2007). Mobile learning: A framework and evaluation. Computers & Education, 49, 581-596. Olney, I., Herrington, J. & Verenikina, I. (2008). iPods in early childhood: Mobile technologies and story telling. In Hello! Where are you in the landscape of educational technology? Proceedings ascilite Melbourne 2008. http://www.ascilite.org.au/conferences/melbourne08/procs/olney.pdf Cheung and Hew 173 Olney, I. & Lefoe, G. (2007). Introducing mobile technologies: Preparatory staff development issues. In ICT: Providing choices for learners and learning. Proceedings ascilite Singapore 2007. http://www.ascilite.org.au/conferences/singapore07/procs/olney.pdf Oliver, B. & Goerke, V. (2007). Australian undergraduates’ use and ownership of emerging technologies: Implications and opportunities for creating engaging learning experiences for the Net Generation. Australasian Journal of Educational Technology, 23(2), 171-186. http://www.ascilite.org.au/ajet/ajet23/oliver.html Pask, G. (1975). Minds and media in education and entertainment: Some theoretical comments illustrated by the design and operation of a system for exteriorizing and manipulating individual theses. In R. Trappl & G. Pask (Eds.), Progress in cybernetics and systems research (pp. 38-50). Washington and London: Hemisphere. Pea, R. D. (1985). Beyond amplification: Using the computer to reorganize mental functioning. Educational Psychologist, 20(4), 167-182. Perry, D. (2003). Handheld computers (PDAs) in schools. Becta ICT research. [viewed 6 Nov 2007, verified 5 Apr 2009] http://publications.becta.org.uk/download.cfm?resID=25833 Polishook, M. (2005). Handheld composing: Reconceptualizing artistic practice with PDAs. In A. Kukulska-Hulme and J. Traxler (Eds.), Mobile learning: A handbook for educators and trainers (pp. 133-138). New York, NY: Routledge, Taylor Francis Group. Ramsden, A. (2005). Evaluating a low cost, wirelessly connected PDA for delivering VLE functionality. In A. Kukulska-Hulme & J. Traxler (Eds.), Mobile learning: A handbook for educators and trainers (pp. 84-91). New York, NY: Routledge, Taylor Francis Group. Reardon, M. (2007). Smartphone sales set to skyrocket. [viewed 21 Nov 2007, verified 6 Apr 2009] http://www.zdnetasia.com/news/communications/0,39044192,62034385,00.htm Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press. Roschelle, J. (2003). Keynote paper: Unlocking the learning value of wireless mobile devices. Journal of Computer Assisted Learning, 19, 260-272. Rosenfeld, P., Booth-Kewley, S., Edwards, J. E. & Thomas, M. D. (1996). Responses on computer surveys: Impression management, social desirability, and the big brother syndrome. Computers in Human Behavior, 12(2), 263-274. Ross, S. M. & Morrison, G. R. (1997). Getting started in instructional technology research (3rd ed.). Bloomington, IN: Association for Educational Communications and Technology. Ross, S. M. & Morrison, G. R. (2004). Experimental research methods. In D. H. Jonassen (Ed.), Handbook of research on educational communications and technology (2nd ed.) (pp 1021-1043). Mahwah, NJ: Lawrence Erlbaum Associates. Schcolnik, M., Kol, S. & Oren, A. (2007). Are handhelds suitable for reading academic texts? In C. Montgomerie & J. Seale (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2007 (pp. 888-895). Chesapeake, VA: AACE. Schubert, R. (2001). Students have music composition in the palms of their hands. Seattle, Washington: Seattle Post-Intelligencer, 20 November 2001. [verified 6 Apr 2009] http://www.seattlepi.com/local/48296_palm27.shtml Schwandt, T. A. (1997). Qualitative inquiry: A dictionary of terms. Thousand Oaks, CA: Sage Publications. Segall, N., Doolen, T. L. & Porter, J. D. (2005). A usability comparison of PDA-based quizzes and paper-and-pencil quizzes. Computers & Education, 45, 417-432. 174 Australasian Journal of Educational Technology, 2009, 25(2) Seppälä, P. & Alamäki, H. (2003). Mobile learning in teacher training. Journal of Computer Assisted Learning, 19, 330-335. Sharples, M., Corlett, D., Bull, S., Chan, T. & Rudman, P. (2005). The student learning organizer. In A. Kukulska-Hulme & J. Traxler (Eds.), Mobile learning: A handbook for educators and trainers (pp. 139-149). New York, NY: Routledge, Taylor Francis Group. Sharples, M., Corlett, D. & Westmancott, O. (2002). The design and implementation of a mobile learning resource. Personal and Ubiquitous Computing, 6, 220-234. Smørdal, O. & Gregory, J. (2003). Personal digital assistants in medical education and practice. Journal of Computer Assisted Learning, 19, 320-329. Soloway, E., Norris, C., Blumenfeld, P., Fishman, B. J. & Marx, R. (2001). Devices are ready-at- hand. Communications of the ACM, 44(6), 15-20. Tallent-Runnels, M. K., Thomas, J. A., Lan, W. Y., Cooper, S., Ahern, T. C., Shaw, S. M., et al. (2006). Teaching courses online: A review of the research. Review of Educational Research, 76(1), 93-135. Tan, N., Chen, W., Looi, C.-K., Zhang, B., Seow, P., Chan, A., Chung, T. M. & Oh, T. T. (2007). Handheld computers as cognitive tools: An environmental learning project in Singapore. In T. Hirashima, U. Hoppe & S. S-C. Young (Eds.), Supporting learning flow through integrative technologies (pp. 377-384). Also in Frontiers in Artificial Intelligence and Applications, 162, 377- 384. Amsterdam, The Netherlands: IOS Press. Thornton, P. & Houser, C. (2005). Using mobile phones in English education. Journal of Computer Assisted Learning, 21, 217-228. Tinker, R. & Krajcik, J. (Eds.) (2001). Portable technologies: Science learning in context. Kluwer Academic/Plenum Publishers, New York. Treadwell, I. (2006). The usability of personal digital assistants (PDAs) for assessment of practical performance. Medical Education, 40, 855-861. Triantafillou, E., Georgiadou, E. & Economides, A. A. (2008). The design and evaluation of a computerized adaptive test on mobile devices. Computers & Education, 50(4), 1319-1330. Tyler-Wood, Y., Rademacher, J., Dunn, L. & Whitworth, C. (2007). Handhelds: An effective assistive technology devices for assisting students with mild disabilities with self-monitoring behavior. In C. Crawford et al. (Eds.), Proceedings of Society for Information Technology and Teacher Education International Conference 2007 (pp. 3654-3659). Chesapeake, VA: AACE. Van ‘T Hooft, M., Diaz, S. & Swan, K. (2004). Examining the potential of handheld computers: Findings from the Ohio PEP project. Journal of Educational Computing Research, 30(4), 295-311. Waycott, J., Jones, A. & Scanlon, E. (2005). PDAs as lifelong learning tools: An activity theory based analysis. Learning, Media and Technology, 30(2), 107-130. Wilkinson, L. & APA Task Force on Statistical Inference (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604. Yamamoto, M. & Akahori, K. (2006). The practice and evaluation of the application of mobile phone in the university class. In C. Crawford et al. (Eds.), Proceedings of the Society for Information Technology and Teacher Education International Conference 2006 (pp. 2440-2450). Chesapeake, VA: AACE. Zurita, G. & Nussbaum, M. (2004). A constructivist mobile learning environment supported by a wireless handheld network. Journal of Computer Assisted Learning, 20, 235-243. Cheung and Hew 175 Appendix: Summary of empirical studies reviewed Author(s) and year Research purpose Research method Data collection method Context Allan (2006) Investigate how students in a middle school in Alberta, Canada together with their teachers and parents perceive the use- fulness of PDAs for student use Descriptive research Student, teacher and parent questionnaires Two middle school classes (one 7th grade, n=26 students; one 9th grade, n=25 students) in Alberta, Canada. In addition, 6 staff members and 24 families participated in the study. Duration of PDA use by students ranged from one year (7th graders) to three years (9th graders). Attewell (2005) Describe the work and key findings of a m- learning research and development project Descriptive research Pre-research individual colla- borating organ- isation project plans and post- research review quest-ionnaires, pre- and post-re- search mentors’ questionnaires, mentor inter- view, pre- and post-mobile learning mentor assessments of their learners’ abilities and attitudes, learners’ text messages. 128 learners from the UK, Italy, and Sweden. Devices included O2 XDA IIs, Sony Ericsson P800s, Sony Ericsson P900s. Examples of use included mobile phone text messaging quizzes, mobile phone sms mini language course. Berglund et al (2007) Describe nurses’ and nurse students’ demands of functions and usability in a PDA Descriptive research Student questionnaire. Student interview Interviews were made with 12 nurses at the County Hospital of Kalmar, Sweden, and a question- naire was given to nurse students (n=84) in their third year at the Dept. of Health and Behavioral Sciences. Duration not reported. Burke et al (2005) Investigate how wireless mobile devices were used both within and outside class, and the specific opportunities and challenges encountered Descriptive research Pre- and post- course student survey responses, faculty participant exit interviews and summative project reports, observations. Food Science & Technology stud- ents utilised PDAs, and temperat- ure probes to gather on site data from grocery stores, restaurants. Biosystems Engineering and Environmental Sci students used PDA based GPS units to collect data in the field for later GIS s/w analysis. Duration of study was two semesters. Number of students not reported. Location Tennessee, USA. PDAs used were Toshiba Pocket PC with internal 802.11b WLAN adapters. 176 Australasian Journal of Educational Technology, 2009, 25(2) Chen & Chung (2007) Explore the use of personalised English vocabulary learning system based on item response theory and learning memory cycle on PDA One group pretest and posttest design Pretest, posttest, student satisfaction questionnaire 15 third year university students majoring in the Dept of English Teaching in Taiwan. Device used was a PDA (HP iPAQ h5550). Duration of study was 5 weeks. The PDAs were used in a personalised English vocabulary learning system based on item response theory and learning memory cycle. Chen et al (2000) Explore the use of handheld for conducting concept tests Descriptive research Student questionnaire 50 students in a chemistry class in the USA. Duration of study was one semester. Device used was HP Jornada 680. The PDAs were used for concept tests – tests which were designed to be taken as part of a lecture with immediate feedback displayed to students and instructor (like a respond pad system). Chen et al (2003) Investigate whether student learning benefited from using mobile devices in a bird watching activity Experiment with random assignment Student pretest on features of birds, middle test (to guide students’ attention to the key features of birds rather than measure their abilities), and posttest. 86 elementary school students in Taiwan. Duration of study one semester (6 bird watching activit- ies). Device used was PDAs with wireless access. The PDAs were used by students (n=42 in 21 pairs) (PDA group) to observe static image of the bird, search for information about birds, and answer multiple questions on features of birds (pretest, middle test, and posttest). Control group (n=44 in 22 pairs) was given a 2- cone telescope and a guidebook. This group identified birds with the telescope, searched for infor- mation with guidebook, received questions orally, and provided answers on a paper worksheet. Churchill & Churchill (2007) Explore the educational affordances of PDAs Descriptive research Teacher interview, teacher reflection logs (blogs), teacher artifacts One teacher from a Hong Kong technical education institute. Duration was six months. Mobile device was a PDA. Cochrane (2008) Investigate the impact of smartphones Descriptive research Pre- and post surveys of students and lecturers, focus group, reflection logs Four courses from an Australian university. Duration ranged from 6 months to 9 months. Mobile devices included Nokia N80 WiFi smart phone, iPod Touch Wifi PDA. Corlett et al (2005) Evaluate a mobile learning organiser for uni students Descriptive research Student questionnaire, logbook, focus group. 17 MSc students at the University of Birmingham, UK. Duration of study was 10 months. Device - was Compaq iPAQ 3760 PDA. Cheung and Hew 177 Dieterle & Dede (2006) Investigate the affordances of wireless handheld devices Descriptive research Student discussion (focus group?) transcripts of the strengths and limits of hand- helds for learn- ing, teaching, and researching. Students from 8 diverse graduate courses (e.g., distributed learning, math, online learning, qualitative methods) in Harvard, USA. Class sizes varied from about 20-50 students. Duration of study was 2 years. Devices used included mobile phones and PDAs. Ferry (2008) Examine pre- service teachers’ use of mobile phones in an environmental education unit Descriptive research Survey of pre- service teachers, reflection data, observation 22 pre-service teachers from a university in Australia. Duration was 6 weeks. Devices used were Palm Treo mobile phones. Franklin & Sexton (2006) Examine faculty use of handhelds in higher education Descriptive research Field observations, faculty journals, and emails 60 faculty members from Colleges of Education and Colleges of Arts & Sciences in Southeastern Ohio, USA. Duration of study 3 years. Devices used were Palm hand- helds such as m130s and Zires. Ganger & Jackson (2003) Explore the use of PDAs for computer-based exams in a wireless environment Descriptive research Student survey 20 freshman medical students in Wayne State Medical School, USA. Duration of study was one semester. Device used was Toshiba PocketPC model e570. Students used the PDAs to answer exam questions, to access web-based course content, for communication, scheduling, as a student response system during lectures, and complete course evaluations. Hennessy (2000) Explore the use of palmtop technology in analysing, graphing, and interpreting numerical data Mixed- method research (qualitative and one- group pretest- posttest design) Student work- sheets, palmtop printouts (text files and graphs), audio recording of whole class disc- ussions and of a target group of 4 boys, observa- tion notes, audio transcripts, pre- post tests, and attitude questionnaire 48 students aged 13-14 in the UK used palmtops. Students worked in groups collecting and graphing temperature data. Duration was 6 sessions of 55 minutes each, over 3 weeks. Subject matter was science. 178 Australasian Journal of Educational Technology, 2009, 25(2) Jackson et al (2005) Investigate the use of wireless handheld computers in the undergraduate medical curriculum Descriptive research Student questionnaire Second and third year medical students in 2003 in the USA. All four classes of medical students in 2004 in the USA. Device used was Toshiba PocketPC e800. The PDAs were used for attendance tracking, course evaluations, interactive learning via class response system, students to send comments, accessing lecture content, tracking patient encounters. Duration of study about 2 years. Johnson & Wilkes (2004) Explore the use of handheld computers Descriptive research Student diary entries, questionnaire 25 students in a senior level information technology class in the USA. Duration of study was one semester. Device used was Dell Axim X5 PDA. Students were assigned a group research project and used the PDA as a collaboration tool. Kennedy et al. (2008) Examine student access to, use of and preferences of technological tools Descriptive research Student questionnaire First year students (n=2,120) at an Australian university. Data were collected during orientation week and the first week of semester 1, 2006. Kong & Li (2007) Examine the effect of using a mobile device supported cognitive tool in learning a primary school math topic Mixed research - one group pretest and posttest design, and descriptive research Pretest, posttest, teacher interview, student questionnaire 36 grade four students in Hong Kong. Duration of whole study not explicitly specified. Device used was a pocket PC. Students used the devices to access a web based cognitive tool Interactive Perimeter Learning for calculating the perimeter of irregular 2-D shapes. Students were able to manipulate (e.g. move) line segments of shape border. Levy & Kennedy (2005) Explore the learning of Italian via mobile phone Descriptive research Student questionnaire and focus group interview 18 uni students in Australia enrolled in a course Italian Literature and Society. Duration of study was 7 weeks. Device used was mobile phones. Students were sent new words, definitions, and example context sentences (Italian vocabulary) at spaced intervals via their mobile phone in between the scheduled lessons and tutorials of their mainstream Italian course. Cheung and Hew 179 Maniar (2007) Explore the effect that screen size has on learning Quasi- experiment Pre- and post- test Each student (n=16) (in the UK) watched an educational 5 minute video on how to measure blood pressure on their allocated devices such as – (a) small screen (Nokia 6600 mobile phone, screen size 42 mm diagonal), (b) medium screen (Motorola E1000 mobile phone, 58 mm), and (c) large screen (Compaq iPAQ H3800 PDA, 96 mm). Before watching the video, participants were asked 4 questions to assess their prior knowledge. After watching the video, they were asked the same 4 questions again. Duration of study not reported. McCracken et al (2007) Explore the contexts and everyday uses of mobile technologies for first year university students Descriptive research Student survey (n=72), student stories, brainstorming notes (n=11) N=11 first year uni. students in Canada who participated in a design workshop. N=72 first year university students in Canada. Duration of the work-shop was 2- hour, while the latter survey was conducted in a 10-min period in an existing class session. Miyata & Kozuki (2007) Evaluate a mob- ile phone supp- orted student response system that enabled pic- tures to be taken and shared Descriptive research Student questionnaire 360 undergraduate students in three different universities in Japan. Duration of study was 4 months in two universities and 3 months in one university. Moallem et al (2003) Examine the effects of wireless handheld technology on the quality of instruction and student learning in a higher educational institute One group pretest and posttest design Student pretest and posttest, student attitude survey, instructor observational notes on students’ engagement in class activities and interactions 52 education major undergrad students in three education courses in the USA. Devices used were PDAs. Duration of study not reported. Instructor used student response system (SRS) during lecture to pose questions and students sent responses via the PDAs. After lecture, students worked in groups on assigned task using PDAs to record their discussion results and send to the instructor via campus email. Motiwalla (2007) Explore the potential role of mobile learning in learning Descriptive research Student survey Study 1: n=19 undergrads in the USA. Duration not mentioned, merely stated the next few weeks of the semester. Mobile device = mobile phone. Students used the phone to access course materials and interact with peers and inst- ructor. Study 2: n=44 undergrads in the USA. Duration 3 weeks. Mobile phones used to access and discuss class materials. 180 Australasian Journal of Educational Technology, 2009, 25(2) Olney & Lefoe (2007) Explore staff development issues relating to the implement- ation of mobile technologies Design based research Workshop evaluation sheets, recorded feedback sessions 12 educators from an Australian university. Devices used were iPod and the Palm Treo mobile phone. Duration of study between 3-5 weeks. Olney et al. (2008) Examine the early childhood pre-service teachers’ use of iPods Descriptive research Reflective journ- als, student interview, observation and products (students’ work) First year students (n~54) in early childhood at an Australian university. Duration of study was 6 weeks. Devices used were iPod. Perry (2003) Evaluate initial issues in the use of PDA type devices in schools both for managing workload and for supporting teaching and learning Descriptive research Participants’ (teachers, students) work logs, telephone survey, interview. 1st phase – 27 schools in the U.K. were equipped with Compaq iPAQs. 16 primary schools, 7 secondaries, 2 infant, one special, and one middle school. 2nd phase – 2 secondaries and 2 primary schools in the U.K. One second- ary and one primary school were given Palm m130s, the others were given iPAQs. Altogether more than 150 teachers used the technologies. Duration of whole study not explicitly reported but believed to be more than 1 year. Polishook (2005) Explore how mobile devices could be used to teach music composition Descriptive research Student weekly evaluations and end of term essays on their experiences using the PDAs. 9 undergraduate and 3 graduate students all majors in music composition in the USA. Duration of study one year. Devices used included Palm VIIx, Palm key- board, an SG-20 MIDI module, and s/w such as NotePad, BeatPad and Theremini. Students wrote music with and for their PDAs and explored how the PDAs might allow for artistic expression that couldn’t be duplicated with other tools. Ramsden (2005) Evaluate if the tools and featur- es of an online learning environment (Blackboard) could be deliver- ed to a wirel- essly connected Palm Pilot Descriptive research Student questionnaires and an end of project interview 13 first-year undergraduate students in the dept of Economics at the University of Bristol, UK. Duration of study was 7 months. Devices used were Palm Pilot m105s. Students used the PDAs to access course documents (e.g., PowerPoint slides, reading lists), participate in discussion boards, and communicate via email. Cheung and Hew 181 Schcolnik et al (2007) Examine whether screen size has effect on reading perfor- mance, student attitudes towar- ds studying from handheld screens, and the transferability of paper reading strategies to digital media Quasi- experiment Baseline test (pretest), posttest, attitude questionnaire 66 first year engineering students from 3 intact classes enrolled in an Advanced English for academic course in Israel. Device used were HP2210 iPAQ and MIO 528. Duration = 4 sessions; however the actual total length was not reported. N=17 in the experiment group (using handheld devices), and n=49 in the control group (using desktop PCs). Segall et al (2005) Compare the effectiveness, efficiency, and satisfaction of a PDA based quiz to that of paper based One group pretest- posttest design Pretest, posttest, time log, and student satisfaction questionnaire. 38 students enrolled in an introd- uctory engineering course in the USA. Device used was PDA (HP Jornada 72). Duration of study was 10 weeks. 34 students took the paper-and-pencil quiz and 29 completed the survey. 30 students took the PDA quiz and 26 completed the survey. Seppälä & Alamäki (2003) Explore mobile learning in teacher training Descriptive research Teacher and student focus group 11 teacher trainees in the Dept. of Home Econs and Craft Science in Helsinki. Mobile device used was a smartphone (Nokia Communic- ator 9210). Duration was one semester. Context in which the phone was used was teacher practicum. Trainees and super- visors used the phones to discuss teaching issues and send SMS messages and digital pictures. Sharples et al (2005) Investigate the use of PDAs to help students manage their studies and learning Descriptive research Student quest- ionnaire, student logbooks (students recorded each use of PDA, the activity, time spent on task, and tools used), student focus groups. 17 Masters students in a Human Centred Systems MSc course in the UK. Study carried out during the academic session 2002/3. Device used was Compaq iPAQ 3760. The PDAs were equipped with a custom designed learning organiser that provided a set of tools for students to access course materials and organise their studies. Smørdal & Gregory (2003) Explore how PDAs could be useful in medical students’ learning Descriptive research Student interviews, walk-throughs, participant observation, video documentation 18 medical students in Norway during clinical practice. Duration was 12 weeks. One group was given PDAs that did not have any direct connection to the Internet. One group had PDAs with wireless connection at 4 locations. One group had PDAs that had GSM cellular phone cards, hence had access to the Internet anywhere. 182 Australasian Journal of Educational Technology, 2009, 25(2) Tan et al (2007) Explore the use of mobile learn- ing technologies for primary environment education in Singapore schools One group pretest- posttest design Pretest, posttest 40 fourth graders in a Singapore school. Pupils divided into groups of 4. Each group has one high, one low achiever, and 2 average students based on the results of their last science assessment scores. Duration of study about 7 hours. Thornton & Houser (2005) Explore the use of mobile phones in English education Mixed method – descriptive research and quasi experiment (not explicitly indicated if the students were randomly assigned into different groups) Survey – descriptive research (n=333 female Japanese university students), pre- and posttest Descriptive research 1: 333 female university students’ ages ranged from 18-21; fields of study included EFL, modern culture, computers, design, and home economics. Descriptive research 2: Three times a day at 9:00, 12:30, and 17:00 hrs, short mini-lessons (less than 100 words of text) were emailed to 44 female Japanese students in two EFL classes. Quasi-experiment 1: 13 students studied 2 sets of messages, with each set consisted of 10 vocabu- lary items over a 2-week period. Lessons were emailed to half the students’ mobile phones, while the other half were encouraged to study identical materials on an identical schedule on the Web. After 2 weeks, the 2 groups switched media for another 2 weeks. Total duration 4 weeks. Quasi-experiment 2: Messages sent to one group’s (n=25) mobile phones, the other group (n=43) were encouraged to study identical messages on paper. Duration 2 weeks. Treadwell (2006) Determine the usability of PDA for assessment Ex-post facto Posttest, time log, student satisfaction questionnaire 3 cohorts of Year 2 medical and dental students in South Africa: cohort 1 assessed using paper based method (2003), n=309; cohort 2 assessed using PDA method (2004), n=314, cohort 3 assessed using PDA method (2005), n=270. Devices included Tungsten E PDAs, Zire 21 PDAs. Triantaf- illou et al (2008) Examine the design and development issues pertaining to the implem- entation of a computerised adaptive test on mobile devices One group pretest- posttest design Pretest, posttest, attitude questionnaire, a debriefing session. 12 students at the 2nd grade of senior high school in Greece. Devices were PDA (HP iPAQ), and Motorola MPx220. Duration of study not indicated. Students first were given a paper and pencil test, after which they completed a test with similar difficulty on the CAT mobile devices. Cheung and Hew 183 Tyler- Wood et al (2007) Explore the use of handhelds as assistive technologies for students with mild disabilities who have issues with self- monitoring Single subject design Student and teachers used a 3-point Likert scale (0=no success with implementation, 1=some success, and 2=great success) 1 student and 3 teachers in the USA. Duration of study was about 8 weeks. Mobile device used was the Dell Axim X30 PDA. The student and teachers used the PDA to record imple- mentation whether the student was successful or not in following the learning expectations (rules) spelt out in the program. Van ‘T Hooft et al (2004) Examine the potential of handheld computers Descriptive research Student and teacher questionnaire 9 teachers (grades 1-12), 217 students in north-east Ohio, USA. Device used Palm IIIc. Subject matter not indicated but appeared to be across many disciplines. Duration of study was 2 weeks. Waycott et al. (2005) Examine the use of PDAs as tools for reading course materials Descriptive research Student questionnaire, interview, transcripts about students’ views of PDAs 65 postgraduate students at an Open University masters course (UK). Device used was PDAs. Duration of study not reported. Yamamoto & Akahori (2006) Examine the application of mobile phone in a university class Descriptive research Student survey, teacher interview 92 students of educational technology in Japan. Duration was one semester. Device used was mobile phone. The phone was used to take students’ attendance, send class inform- ation to students, for students to send in their homework via the phone’s email, and for students to send their opinions regarding issues raised in class. Zurita & Nussbaum (2004) Examine a learning environment supported by handhelds to teach reading for first graders Experiment Pre- and posttest to measure children’s know- ledge of word construction, observation, and interview N= 12 first graders (6-7 years old) in Chile. Duration 4 weeks. Handheld used to construct words from syllables. Dr Wing Sum Cheung (author for correspondence) Associate Professor, Learning Sciences & Technologies National Institute of Education, Nanyang Technological University 1 Nanyang Walk, Singapore 637616, Singapore Email: wingsum.cheung@nie.edu.sg Dr Khe Foon Hew, Assistant Professor, Learning Sciences & Technologies National Institute of Education, Nanyang Technological University 1 Nanyang Walk, Singapore 637616, Singapore Email: khefoon.hew@nie.edu.sg