Australasian Journal of Educational Technology, 2019, 35(1). 85 Saudi undergraduate students’ perceptions of the use of smartphone clicker apps on learning performance Abdulaziz Aljaloud, Nicolas Gromik, Paul Kwan, William Billingsley University of New England This study aimed to investigate how the use of a smartphone clicker app by a group of 390 Saudi Arabian male undergraduate students would impact their learning performance while participating in a computer science class. The smartphone clicker app was used by the students during peer group discussions and to respond to teacher questions. A conceptual framework identified teacher-student and student-student interactions, collaborative learning, and student engagement as three primary practices that could improve student performance when a smartphone clicker app was used. The relationships between these factors were tested empirically by participant completion of a self-administered online survey. This study found the use of a smartphone clicker app promoted increased teacher-student and student-student interactivity, leading to active collaboration learning by students and improved learning performance. No positive relationship was found between the smartphone clicker app use and increased student engagement. These results demonstrated the role of the smartphone clicker app in enhancing the learning experience of the Saudi undergraduate students included in this study, but not the overall student engagement. Further research into how use of a smartphone clicker app in classroom settings might promote student engagement to improve the overall learning performance is needed. Introduction Smartphone clicker apps for learning are increasingly considered for use in education settings to increase teacher-student and student-student interactivity, enhance student engagement, and improve student understanding of the learning content (Lopez, Love, & Watters, 2014). As a result, there is a growing body of resources available to teachers at all levels of education which is designed to facilitate the use of clicker apps in the classroom (Bojinova & Oigara, 2011). Notwithstanding the research evidence to date – derived primarily through the application of qualitative research paradigms – which demonstrates the potential for clicker apps to effectively transform didactic teacher-centred learning settings into interactive student-centred learning settings (e.g., Hwang, Wong, Lam, & Lam, 2015), there remains only limited research understanding of the mechanisms and processes underpinning the technology’s impact on the learning performances of students. In response to the need to further develop a research understanding of the relationship between clicker technology use for learning and student learning outcomes, this study attempted to repeat the key study by Blasco-Arcas, Buil, Hernandez-Ortega, and Sese (2013), but in a different context; namely a higher education institution in Saudi Arabia. Blasco-Arcas et al. (2013) provided a key study in clicker literature through the development and application of their conceptual framework to explain the mechanisms through which clicker technology impacts student learning practices and outcomes. In turn, replicating the methodology of this study in the Saudi higher education context on male students only is of research importance as it will contribute to academic understanding of the implications of cultural and gender factors for mobile technology use for learning and for the learning outcomes of students. Smartphone The focus of this study was on how a smartphone clicker app might support the delivery of lessons on mobile information technology (IT) and improve teacher-student communication. Recent survey research on mobile IT in higher education show that students are the main driving force behind the use of mobile IT devices in higher education, with most believing that these devices contribute to their academic successes across different subject areas (Gikas & Grant, 2013; Kim, Rueckert, Kim, & Seo, 2013; Tai, 2012). There is however research Australasian Journal of Educational Technology, 2019, 35(1). 86 evidence to show that numerous issues still exist regarding the adoption of mobile technology platforms for learning. For instance, in their study of 160 British university students’ use of digital technologies, Margaryan, Littlejohn, and Vojt (2011) found low-level use of, and familiarity with, collaborative knowledge creation technologies by students, and that students’ attitudes towards technology use for learning appear to be limited to minor utilisation of technology tools for content delivery. Age differences were also found to be a significant factor impacting technology use. In addition, there is evidence to suggest that students believe the available mobile technologies are largely unable to support ownership of learning to their expectations (Kukulska-Hulme, 2009), and that many educators continue to face difficulties in developing new teaching and learning methods that demonstrate innovative technology-based learning (Kim et al., 2013). The widespread use of information and communications technology (ICT) on higher education campuses has the potential to generate new learning opportunities for students as well as enhance and expand the instructional strategies utilised by teachers (Blasco-Arcas et al., 2013). More and more, mobile IT devices are providing students with the opportunity to increase their participation in learning activities, interacting with teachers and fellow students, and accessing course content more quickly and easily (Cavus & Ibrahim, 2009; Nihalani & Mayrath, 2010). The learning opportunities supported by mobile IT devices will only be further enhanced if the devices are combined with social media platforms and other web-based tools into the future (Rodriguez, 2011). Traditional clicker Overview Clickers, also known as instant response systems (IRS), students response systems (SRS) or audience response systems (ARS) are increasingly integral to teaching and learning practices in some educational settings. By 2004, clickers have been incorporated into more than 1 million schools and universities worldwide. This figure increased to 8 million by 2008 (Chien, Chang, & Chang, 2016). Traditional clickers were used to collect student responses to questions posed by the teacher (e.g., multiple choice questions) which were sent to a monitoring system via infrared or radio frequency signals (Chien et al., 2016). What were once essentially handheld hardware-software polling systems have become clicker applications, including i>clicker GO and Socrative, which support mobile devices and a range of internet platforms such as GoSoapBox and QuestionPress. Hence, classical clicker systems have developed into more sophisticated web- based alternatives that are becoming more and more pervasive in higher education learning activities and other educational settings (Richardson, Dunn, McDonald, & Oprescu, 2015). Benefits of clicker-based learning The benefits of the clicker for teacher-student interactions and student participation rates more broadly is that all students can express their thoughts/answers without the fear of being scrutinised by peers. The clicker also provides teachers with an aggregation of the students’ responses via histograms and the option to whether to publicly identify specific students’ responses or a student group’s responses as a whole (Chien et al., 2016). Student anonymity when responding to teacher questions is regarded as the main advantage of the clicker. Given the opportunity to remain anonymous, students will potentially increase their level of participation in the learning activity by providing their thoughts and insights (Bousbahi, 2014; Castillo-Manzano, Castro-Nuno, Lopez-Valpuesta, Sanz-Diaz, & Yniguez , 2016). In addition, the independent nature of the clicker is considered a useful element for reducing both conformity in the classroom and the anxiety sometimes experienced by students when asked to contribute in a public forum such as a classroom full of their peers (Bojinova & Oigara, 2013). Use of clickers is associated with more sustained learner engagement. Studies show that teachers can use clicker questions to initiate class discussions and get students to refocus their attention by intermittently interjecting clicker questions into the learning sequence (Zhu, 2008). Similarly, it has been reported that students’ perceptions on the use of clickers in the classroom is an effective way to stimulate their interest in, and Australasian Journal of Educational Technology, 2019, 35(1). 87 engagement with, the topic being taught in the classroom (Bergtrom, 2006; Simpson & Oliver, 2007). A study of clicker use for student polling (responses to multiple-choice questions) in undergraduate classes conducted by Calma, Webster, Petry, and Pesina (2014) reported that clicker use improved both student interaction and engagement in class. Female students particularly reported feeling more engaged in the lesson as a result of the use of the clicker by the lecturer to provide immediate feedback and to facilitate deeper level understanding. Another benefit of the clicker is that it provides a useful mechanism to quickly and easily access and represent information. This is an important attribute as it supports the provision of timely feedback to students regarding the extent to which key concepts and aspects of the content have been understood or applied. Teachers can thus identify which content elements require additional emphasis to facilitate further understanding (Vana, Silva, Muzyka, & Hirani, 2011). Lastly, clickers have been utilised by teachers for practical outcomes such as recording class attendance. Indeed, clicker use is associated with higher class attendance rates by students, especially when attending class is tied to grades (Hunsu, Adesope, & Bayly, 2016). Challenges associated with clicker-based learning The use of clickers is not without its challenges. Teachers report an increase in preparation time required to introduce clicker use into the learning activities (Keogh & Wang, 2010). In addition, there are challenges associated with the costs of the technological infrastructure required to support clicker use and the potential changes required to course syllabuses. There is also the ongoing challenge for learning institutions to keep pace with technological advances to ensure their use remains relevant and engaging for students. Indeed, recent research suggests that not only do “instructors face the difficult task of controlling digital distractions within the classroom” (Pistilli & Cain, 2016, p. 247), but such distractions may lead students to be less able to use their newly acquired knowledge in unfamiliar situations (Morrell & Joyce, 2015). Research focus Context Culture consists of the norms, beliefs and customs that underpin how family, education and social systems are constructed (Al-Jumeily, Hussain, & Crate, 2014; Tweed & Lehman, 2002). Research affirms the relationship between cultural factors and technology uptake by citizens (e.g., Barton 2010; Hofstede 1980; Tse, Tiong, & Kangaslahti, 2004). Notably, most ICT systems have been developed in western countries and therefore contain a cultural bias in in relation to the salient cultures of those societies (Al-Jumeily et al., 2014). Hence, some new technologies may not be easily utilised or considered appropriate for non-western cultures. Culture can thus significantly influence how learners utilise and respond to ICTs, including software apps and the internet (Arenas-Gaitán, Ramírez-Correa, & Rondán-Cataluña, 2011). As Van Raaij and Schepers (2008) explain, ICT system features affect adoption process as they affect the nature of the user’s personal interaction with the technology and invariably raise issues related to gender, learner motivational, access to technical support, and perceived compatibility. Technology remains important and integral to social operations in the Kingdom of Saudi Arabia. This is evidenced in the rapid increase in internet usage throughout the kingdom from only 200,000 people in December 2000 to more than 16.5 million users in 2013 (Communications and Information Technology Commission, 2014). Technology enhanced learning has demonstrated progress and development across all levels of education in Saudi Arabia in recent decades (Bousbahi, 2014). This is due in part to generous government spending on education in the kingdom, which was SR 204 billion in 2013 (Irfan, 2013). In terms of the higher education sector and technology-based learning, the internet is a vital tool for the development of Saudi students’ knowledge and skills. The size of the higher education sector has increased considerably during the previous 50 years. There are presently 25 public sector universities and many private sector universities throughout the kingdom. Australasian Journal of Educational Technology, 2019, 35(1). 88 Higher education teachers are expected to increasingly use IT-based tools in their pedagogy, as reflected in initiatives by the Ministry of Education such as the National Plan for Information Technology. This plan aims to improve technology-based educational systems in higher education institutions and to promote e-learning and distance learning at the higher education level. Notably, research confirms that Saudi students welcome the adoption of new technologies for mobile learning (e.g., Al-Emran, Elsherif, & Shaalan, 2016). In particular, Saudi students were reported to like the flexibility afforded to learning processes by mobile learning devices and platforms, along with faster and easier access to learning materials and improved communication channels between teachers and their peers (Chanchary & Islam, 2011). Despite continued developments in the form, capabilities, and availability of clickers for educational settings, no Saudi university currently utilises clickers in the classrooms. The increasing popularity of mobile devices in Saudi Arabia supports the argument that clicker apps in mobile IT devices, such as smartphones, may have an important role to play in educational service delivery in Saudi Arabia in the future. Indeed, there are universities in Saudi Arabia that have introduced mobile-based learning activities into some learning contexts. Moreover, mobile device apps that include features such as campus maps, calendars, e-service locations, and university news and events bulletins are being used, in addition to being used for learning management systems. However, some Saudi universities still lag behind the trend and there has been little academic research to date to provide insights into what mobile IT initiatives are being undertaken at the universities and how they may optimise mobile IT based learning. Conceptual framework, research question, and hypotheses The conceptual framework (Figure 1) introduced by Blasco-Arcas et al. (2013) was applied in this research to explain how use of a smartphone clicker app might enhance the learning performances of undergraduate students at a major university in Saudi Arabia. The key mechanisms to impact student learning performance identified in the framework are interactivity, active collaborative learning, and student engagement. Interactivity is widely regarded as vital to effective learning on the basis that the platform provides for the exchange of ideas and opinions and the general support for participative learning (Richardson et al., 2015). There are, however, clear implications for interactivity, particularly the promotion of teacher-student interactivity in large classroom settings such as the university’s lecture rooms. Figure 1. Conceptual framework used to investigate smartphone clicker app use to enhance student learning performance (Blasco-Arcas et al., 2013) Interactivity with students is characterised as general participation in the processing of learning materials and the exchange of ideas using the clicker app; whereas interactivity with the teacher is characterised as use of the clicker app to access students’ understandings of the learning content and to provide feedback (Blasco-Arcas et al., 2013). Active collaborative learning is characterised as use of the clicker app to facilitate intentional participation throughout the learning activity by teachers and students, to share ideas and information, and to support topic clarification and the generation of new knowledge (Chuang, 2015). Technology has long been regarded as a potentially effective facilitator of active collaborative learning (Rodriguez, 2011), and in this study the clicker app is utilised by the students as a means of receiving questions posed by the teacher, and Australasian Journal of Educational Technology, 2019, 35(1). 89 providing responses that contribute to knowledge creation. Engagement is characterised as the students’ involvement in the learning activity based on how they perceive their interactions with other students and the teacher. Student engagement is understood to mediate the impact of curriculum design and implementation on student learning performance (Kahu, 2013). Hence, it is an important mechanism to focus on because it implies use of appropriate cognitive processes by students to achieve learning success (Blasco-Arcas et al., 2013). Indeed, high engagement by a student is associated with increased motivation, commitment, and enhanced overall performance (Kahu, 2013). Lastly, a high level of student engagement is also indicative of a dynamic and rich learning environment in which students are motivated to actively contribute to improve their learning performance (Beauchamp & Kennewell, 2010). The Blasco-Arcas et al. (2013) conceptual framework was tested empirically through use of the smartphone clicker app in peer group discussions and student responses to teacher questions. The aim was to promote collaborative work, and foster student interactivity and engagement, respectively. The research question to be answered was: What are the effects of the smartphone clicker app use on the learning performance of undergraduate students in Saudi Arabia? Based on the preceding discussion, this research investigation speculates that interactivity (teacher-student and student-student) contributes to student engagement with the learning content, as well as to active collaborative learning among students. To test the importance of these two consequences of interactivity, the following seven hypotheses were formulated based on the conceptual framework presented in Figure 1. H1: Student interaction with other students via use of a smartphone clicker app will increase student collaborative learning activity; H2: Student interaction with other students via use of a smartphone clicker app will increase student engagement; H3: Student interaction with teachers via use of a smartphone clicker app will increase student collaborative learning activity; H4: Student interaction with teachers via use of a smartphone clicker app will increase student engagement; H5: Student engagement via use of a smartphone clicker app will improve student learning performance; H6: Student collaborative learning activity via use of a smartphone clicker app will improve student engagement; and H7: Student collaborative learning activity via use of a smartphone clicker app will improve student learning performance. Methodology The objective of this study was to replicate, and also to re-contextualise, the methodology applied by Blasco- Arcas et al. (2013) in their novel study of the impact of clicker app technology on student interactivity, collaborative learning, and engagement in learning performance. A key strength of their exploratory quantitative study was the development and use of a conceptual framework to improve research knowledge of the key mechanisms linking clicker technology to student learning outcomes. Moreover, the use of a survey questionnaire to collect data from 198 college business students in Spain provided a robust tool to better understand the nature of college students’ relationship with clicker technology in the context of learning performance. It is noted that exploratory quantitative studies are typically not undertaken to draw definitive research conclusions due to their lack of statistical strength (Johnson & Christensen, 2013), as evidenced by their being no use of engagement statistics by Blasco-Arcas et al. (2013) when measuring student engagement. Nonetheless, studies to improve researcher knowledge in this field are important to higher education policy makers, university administers, and teachers as they provide research evidence to better comprehend the why and how factors in the relationship between technology for learning (e.g., clicker technology) and student learning outcomes. Australasian Journal of Educational Technology, 2019, 35(1). 90 This study was conducted at the University of Hail, Saudi Arabia during the second semester of 2015 with a sample of 390 male undergraduate students enrolled in computer science classes. To justify the use of an all- male sample, it is noted that Islam strongly endorses mandatory education for both males and females. The goals of Saudi education programs, as embodied in the Shari'a are to “instill in Saudi society a particular vision of the moral and religious life” (Baki, 2004, p. 1). Within Shari'a orthodoxy, there are different social expectations placed on males and females which dictate that they be assigned to different educational pathways and to different educational programs (Baki, 2004). Procedure At the beginning of the semester, the students were notified that they would be presented periodically with practice questions using a smartphone clicker app. The students were introduced to the clicker app by the researcher and their teachers. They were also given installation and use instructions. They were then asked to use the smartphone clicker app at three times during the lesson: (1) during a quiz activity at the beginning of the lesson to review previous lessoning and to assess their retention of the main themes and concepts; 92) during the course of the lesson to review new concepts and promote engagement; and (3) during the break in the lesson to give them the opportunity to discuss the learning activities. Data collection A self-administered online survey was sent to 600 students who participated in the smartphone clicker app use experience. Completion of the survey was voluntary and student anonymity was maintained at all times. Students responded to the survey items via a 5-point Likert scale (1 = strongly disagree to 5 = strongly agree). The survey items also collected demographic information. Of the 600 surveys distributed, 413 were returned, 23 of which were incorrectly or inadequately completed, leaving 390 valid responses for data analysis. Table 1 shows the 16 survey items used to measure the five domains embedded in the conceptual framework; namely, interactivity with students, interactivity with teachers, active collaborative learning, student engagement, and learning performance. Analysis and results The study utilised both SPSS™ and AMOS™ (an extension of SPSS™) for data analysis. Exploratory factor analysis and descriptive statistics were applied using SPSS™. AMOS™ was utilised to test whether the data fitted the model. Measurement models were developed to ascertain validity, and structural models were designed to test the hypotheses. Confirmatory factor analysis Initially, an exploratory factor analysis (EFA) was conducted to highlight the factor structure. The EFA revealed all items loaded onto their respective factors. Notably, two items were deleted because they failed to attain the minimum factor loading of .50. A confirmatory factor analysis (CFA) was subsequently performed. Results revealed that the model exceeded the optimal levels of goodness-of-fit indices suggested by Hair, Black, Babin, Anderson, and Tathan (2006): NNFI (TLI) = .987; NFI = .975; IFI = .990; CFI = .990; RMSEA = .041; Normed (CMIN) = 1.639. The CFA results offered powerful support for the assumption of the appropriateness of the measurement model for the data. According to standards defined by Falk and Miller (1992), a construct is considered acceptable for all loadings with values of more than 0.50. The factor loadings for each item were consistent with necessary conditions. The values are given in Table 1. Australasian Journal of Educational Technology, 2019, 35(1). 91 Table 1 Item results Construct Items Standardised factor loading (> .505) Interactivity with students Using the smartphone clicker app facilitated interaction with other students (P1). .814 Using the smartphone clicker app in this class provided me the opportunity to engage in a discussion with my peers (P2). .887 Using the clicker app facilitated dialogue with other students (P3). .836 Interactivity with teachers Using the smartphone clicker app in this class allowed the exchange of information with other students (T1). .894 Using the smartphone clicker app facilitated interaction with the teacher (T2). .931 Using the smartphone clicker app in this class gave me the opportunity to discuss with the teacher (T3). .904 Active collaborative learning Using the smartphone clicker app allowed me to actively collaborate within my learning experience (act1). .928 Using the smartphone clicker app allowed me to collaborate with others on the development of my learning experience (act2). .934 Using the smartphone clicker app allowed me to have free rein to co-create my own learning experience (act3). .953 Using the smartphone clicker app allowed me to participate freely in my learning experience (act4). .891 Engagement The smartphone clicker app led to my opinions being taken into account in this class (eng1). .927 The smartphone clicker app spurred interactions with faculty and peers that made me feel valued (eng2). .934 The smartphone clicker app improved the relationships I had with other students and the teacher (eng3). .893 Learning performance Using the smartphone clicker app helped to improve my ability to comprehend the concepts in this module (L1). .927 Using the smartphone clicker app enhanced my learning experience in this module (L2). .927 Using the smartphone clicker app enabled me to gain a better understanding of the key concepts being taught (L3). .871 Each item’s internal coherence was measured on the basis of reliability. To judge the internal coherence of the data, two coefficients were used to verify the reliability of the items of this study: Cronbach’s alpha coefficient which measures the contribution of each item in terms of similarity; and Werts, Linn, and Jöreskog (1974) composite reliabilities coefficient which measures the value of the respective items. As detailed in Table 2, the values of each of these two coefficients were higher than 0.70, the recommended value for research according to Churchill (1979). Convergent validity – the degree of relation between the measures of two constructs – was measured by average variance extracted (AVE), which indicates the common variance between an item and its respective construct (Escobar-Rodriguez & Monge-Lozano, 2012). For AVE, an acceptable threshold is greater than 0.50 (Hair et al., 2006). The AVE scores for each of the five constructs employed in this study exceeded this minimum value (Table 2). Australasian Journal of Educational Technology, 2019, 35(1). 92 Table 2 Composite reliability coefficient (CRC) and Cronbach’s alpha coefficient and average variance CRC (> .70) Cronbach’s alpha Average variance (> .50) Interactivity with students 0.883 .741 0.716 Interactivity with teachers 0.935 .890 0.826 Engagement 0.942 .933 0.844 Active collaborative learning 0.961 .930 0.859 Learning performance 0.935 .898 0.827 Discriminant validity assesses whether the constructs of a study hypothesised to be unrelated are indeed not related, and is confirmed if the square root of the AVE is of a higher value than the correlation between the constructs (Zait & Bertea, 2011). The square roots of AVE values are listed in Table 3, indicated by the diagonal line of boldface numbers. As the square roots of the AVE were universally larger than the correlation between the constructs in the results of this study, adequate discriminate validity was achieved. Lastly, the confidence intervals surrounding the correlation estimates between any two factors were tested; discriminant validity was supported since no intervals included the value of 1.0 (Anderson & Gerbing, 1988). Table 3 Correlation Interactivity with students Interactivity with teachers Engagement Active collaborative learning Learning performance Interactivity with students 0.846 Interactivity with teachers 0.397 0.909 Engagement 0.207 0.512 0.918 Active collaborative learning 0.507 0.555 0.374 0.927 Learning performance 0.264 0.285 0.089 0.286 0.910 Structural model The hypotheses of this study were substantiated with a structural model (Figure 1). The results of the structural model generated an overall good fit: NFI = .968, NNFI (TLI) = .980, CFI = .984, IFI = .984, RMSEA = .051, and Normed (CMIN) = 1.998. Hypothesis 1 stated that student interaction with other students via the use of a smartphone clicker app would increase student collaborative learning. The data analysis showed a significant positive relationship between interactivity with students and collaborative learning (β1 =.473, p < .05), so H1 was supported. This significant effect indicated the prominence of promoting interactivity among students to encourage active collaborative learning in class via use of a smartphone clicker app. Hypothesis 2 stated that student interaction with other students via the use of a smartphone clicker app would increase student engagement. This study found an insignificant relationship between interactivity with students and engagement (β2 = .029, p > .1), so H2 was not supported. Therefore, the significance of stimulating interactivity among students did not increase the level of student engagement in this study. Hypothesis 3 stated that student interaction with teachers via the use of a smartphone clicker app would increase student collaborative learning. The data analysis showed that collaborative learning was positively and significantly influenced by interactivity with teachers (β3 =.184, p < .05), so H3 was supported. This result also highlighted the importance of promoting student interactivity with teachers to increase active collaborative learning. Hypothesis 4 stated that student interaction with teachers via the use of a smartphone clicker app would increase student engagement. The effect of student interactivity with teachers on student engagement Australasian Journal of Educational Technology, 2019, 35(1). 93 was found to be insignificant (β4 = −.020, p > .1), so H4 was not supported. The initiative to promote interactivity between students and teachers as a part of the learning activity did not enhance student engagement. The study also evaluated how student engagement affects student learning performance (H5). The results indicated that H5 was supported (β5 = .354, p <.05). Hypothesis 6 stated that student collaborative learning via the use of a smartphone clicker app would improve student engagement. The data analysis found a significant association between active collaborative learning and student engagement (β6 = .354, p < .001), so H6 was also supported. This result further highlighted the need for collaborative learning to stimulate student engagement. Finally, Hypothesis 7 stated that student collaborative learning via the use of a smartphone clicker app would improve student learning performance. This study found support that student learning performance was positively and significantly affected by student collaborative learning, which offered support for H7 (β6 = .421, p <.05). Figure 2. Results of testing model: Amos output Discussion This study found that the smartphone clicker app functioned effectively as a platform for student-student and teacher-student interactivity during class activities. In turn, undergraduate students’ active collaborative learning increased due to the interactivity (as evidenced in the results’ support for hypotheses 1 and 3). The students regarded interactivity via the smartphone clicker app as a useful pathway for receiving and evaluating information. Hence, the results suggest use of smartphone clicker apps in class increases Saudi male undergraduate students’ active collaborative learning practices with both their peers and the teacher. This finding aligns with those reported by Bousbahi (2014) and Blasco-Arcas et al. (2013). However, this study also found that use of the smartphone clicker app for interactivity did not increase the students’ level of engagement in the learning activity (as evidenced in the failure of the results to support hypotheses 2 and 4). This finding is in contrast to those reported in previous studies on traditional clicker system use in education settings by Calma Australasian Journal of Educational Technology, 2019, 35(1). 94 et al. (2014) which, for example, reported a correlation between clicker use in higher education and an increase in student engagement. Two explanations may account for the difference in these findings. First, the use of the smartphone clicker app may not have led to an increase in student engagement because a smartphone clicker app was used compared to a traditional clicker app used by Calma et al. (2014). Although use of mobile technology for learning is increasingly prevalent in higher education settings, research evidence points to ongoing issues regarding the impact of mobile phone use on student engagement and overall learning outcomes. For instance, Duncan, Hoekstra, and Wilcox (2012) found that use of mobile phones in class was significantly correlated to a fall in undergraduate student grades. According to the authors, the negative correlation was primarily due to the inability of the students to effectively multitask using the technology. More recently, Lepp, Barkley, and Karpinski (2015) reported a significant negative correlation between increased mobile phone use and decreased academic performance (GPA scores) in undergraduate students in the United States. The main issue was the inability of the undergraduate students to properly differentiate between the leisure and academic use of the mobile phone. Therefore, in this study it may have been that the students were enthusiastic about interacting with the teacher using their mobile phone, but this enthusiasm did not translate to greater engagement with the learning content. The second explanation for the finding in this study that the smartphone clicker app use did not increase student engagement may be related to the segregated gender higher education context in Saudi Arabia. Gender has historically been an important factor in the digital divide in relation to the utilisation of certain types of technology (Cotten, Shank, & Anderson, 2014). Males have been found to typically exhibit greater use of mobile phones for recreational purposes such as gaming (Cotten, Anderson, & Tufekci, 2009); and females have been found to typically exhibit a greater propensity to share creative digital content online (Hargittai & Walejko, 2008). In relation to mobile technology use for learning specifically, previous research evidence (e.g., Calma et al., 2014) indicates female undergraduate students perceive the use of clicker in class helps them to feel they are “more prepared for and engaged in class” (p. 114) as compared to their male counterparts. In addition, Junco, Merson, and Salter (2010) found females are generally more prolific users of digital technology than males when engaging in academic activities. Notwithstanding these findings, the research evidence on the implications of gender for mobile technology use for learning is far from conclusive. The study by Al-Emran et al. (2016), for example, of 383 students studying in Oman and UAE found no statistical difference between male and female students in their attitudes towards mobile device use for learning. The sample in this study comprised males due to gender separation in Saudi higher education institutions. Thus, the result that the smartphone clicker app use did not increase student engagement may reflect a form of gender divide in relation to technology and engagement reported in research studies. Furthermore, it may be the case that the limited use of a clicker app in university classrooms implies undergraduate students are primarily focused on – or distracted by – the novelty of the interactivity mode. Access to mobile devices in learning settings has the potential to result in teachers having to compete for students' attention among a suite of communication modes (Sana, Weston, & Cepeda, 2013). Moreover, Kuznekoff, Munz, and Titsworth (2015) reported in their study of 47 college students in America that students who did not have access to and use of a mobile device during lessons generated up to 62% more information in their notes, recalled more detailed information from the lecture, and achieved higher multiple-choice test scores than students who had access to and actively used a mobile phone in class. Thus, the relationships between Saudi students’ technology use, engagement, and academic performance are complex and remain an important focus for ongoing research investigations (Rashid & Asghar, 2016). As such, there are clear implications for the choice of strategies undertaken by Saudi university administrators – at both male and female campuses – to appropriately leverage the use of mobile devices by students through policy regulation and lesson design to optimise the learning outcomes for students as a result of mobile device use (Parcha, 2014). Australasian Journal of Educational Technology, 2019, 35(1). 95 Implications for practice Higher education policy makers, curriculum designers, and university teachers in Saudi Arabia continue to acknowledge and respond to the growing body of research evidence that sheds light on the relationship between mobile devices for learning and student learning outcomes. Appropriate progress in this regard however relies on clearly defined policy initiatives, the provision of technical and pedagogical support to teachers, and the implementation of university-based regulations on student uses of technology in the classroom. This study found that the smartphone clicker app facilitated student-student and teacher-student interactivity during class activities, and promoted active collaborative learning, but did not increase the students’ level of engagement in the learning process. The implications for practice point to the need for the selection and implementation of appropriate strategies for the integration of smartphone clicker technology into the classrooms. Importantly, these strategies must address the potential for gender variables to influence how the Saudi undergraduate students perceive the use of such technologies to improve their learning performance. There is emerging research evidence which points to greater learning engagement from the use of e-learning technology among female Saudi university students. For instance, Alshawi and Alhomoud (2016) studied the effects of Edmodo use on EFL college student learning; with an all-female cohort. With focus on engagement in learning, learning motivation, and communication with teachers as the key learning elements, the authors reported that the integration of Edmodo into the learning activities promoted EFL student engagement in the learning activities and increased their willingness to incorporate technology into their interactions with teachers during the language learning process. In addition, other research studies of Saudi female university students report a positive relationship between technology use and skills proficiencies. For example, Alfaleh (2017) sought to identify the extent to which students at Princess Nourah bint Abdulrahman University (an all-female cohort) had acquired the e-learning competencies required to qualify them for the Saudi labour market. Notably, the author found that the female students generally possessed high-level skills to effectively utilise the internet and medium-level attitudes towards an e-learning culture, leading to the conclusion that the university would achieve further positive outcomes for the students through the development of IT educational courses and training programs. Limitations Notwithstanding the valuable insights into the complex interrelationships between smartphone clicker app use and undergraduate student learning performance that have emerged from the findings of this research, this study is limited by its focus on Saudi male undergraduate students only. Future studies may therefore contribute further insights into the nature of the interrelationships through a comparative analysis of learning performance outcomes for Saudi female undergraduate students following their smartphone clicker app use in class. Furthermore, the self-report data collection method to measure student engagement reduces the generalisability of the results. Conclusion Higher education institutions in all countries around the world have a vested interest in gaining a better understanding of how technology can potentially improve student learning performance. As such, there has been increasing research interest in the ways clickers can be used to complement current pedagogical practices or facilitate new instructional strategies in higher education settings (Chien et al., 2016). In particular, focus has been given to an examination of the role that developing smartphone clicker app technology plays in shaping both how students learn and their level of engagement during the learning process. This study applied a conceptual framework to improve academic awareness of the interrelationships between smartphone clicker app technology use in the classroom and student interactivity (teacher-student and student-student), collaboration and engagement, as crucial academic behaviours of learning performance. The results that emerged from this research were discussed in relation to the key assumptions and findings in the existing literature. The results affirmed the growing and compelling research evidence that shows the beneficial role that the smartphone clicker app – and mobile IT devices more generally – can play in increasing student participation during lessons and in facilitating collaborative learning practices by students. Furthermore, the Australasian Journal of Educational Technology, 2019, 35(1). 96 results reflected other research findings that the use of such technologies in the classroom may not provide the benefits of learning engagement to male students where compared to their female counterparts. In light of these findings, academic understanding of the nature of the relationship between mobile technology use and student engagement may be enhanced by further research that applies actual indices of engagement via researcher’s observation – rather than self-reported outcomes – as a more direct way to determine gender differences in engagement behaviours. Future research may also focus on a comparative investigation of the smartphone clicker app use and student engagement in different types of courses in order to gain a better understanding of whether or not course type influences student engagement outcomes. Lastly, it is recommended that conceptual frameworks should continue to be utilised in empirical investigations to further analyse how technology can be used in higher education settings as an educational tool in order to improve learning performance in relation to student engagement, particularly amongst the current generation of technology-driven students in Saudi Arabia. References Al-Emran, M., Elsherif, H. M., & Shaalan, K. (2016). Investigating attitudes towards the use of mobile learning in higher education. Computers in Human Behavior, 56(C), 93-102. https://doi.org/10.1016/j.chb.2015.11.033 Alfaleh, M. A. (2017). Extent of e-learning competencies by students of Princess Nourah bint Abdulrahman University necessary for accessing the labour market. Asian Education Studies, 2(2), 27-39. Al-Jumeily, D., Hussain, A., & Crate, S. (2014). The impact of cultural factors on technology acceptance, student’s point of view. In Proceedings of the International Conference on Frontiers in Education: Computer Science and Computer Engineering (FECS), Las Vegas, NV., 1-7. Alshawi, S. T., & Alhomoud, F. A. (2016). The impact of using Edmodo on Saudi university EFL students' motivation and teacher-student communication. International Journal of Education, 8(4), 105-121. Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103(3), 411-422. Anderson, T. (2003). Modes of interaction in distance education: recent developments and research questions. In M. D. Moore and W. G. Anderson (Eds.), Handbook of distance education (pp. 129-144). Mahwah, NJ: Lawrence Erlbaum Associates. Arenas-Gaitán, J., Ramírez-Correa, P. E., & Rondán-Cataluña, F. J. (2011). Cross cultural analysis of the use and perception of web based learning systems. Computers & Education, 57(2), 1762-1774. Baki, R., (2004). Gender-segregated education in Saudi Arabia: Its impact on social norms and the Saudi labor market. Education Policy Analysis Archives, 12(28), 1-17. Barton, S. (2010). Social and cultural factors that influence the uptake of e-learning: Case studies in Malaysia, Indonesia, Turkey, Singapore and Australia. (Doctoral dissertation). RMIT, Melbourne. Beauchamp, G., & Kennewell, S. (2010). Interactivity in the classroom and its impact on learning. Computers & Education, 54(3), 759-766. https://doi.org/10.1016/j.compedu.2009.09.033 Bergtrom, G. (2006). Clicker sets as learning objects. Interdisciplinary Journal of Knowledge and Learning Objects, 2(1), 105-110. Blasco-Arcas, L., Buil, I., Hernández-Ortega, B., & Sese, F. J. (2013). Using clickers in class. The role of interactivity, active collaborative learning and engagement in learning performance. Computers & Education, 62, 102-110. https://doi.org/10.1016/j.compedu.2012.10.019 Bojinova, E. D., & Oigara, J. N. (2011). Teaching and learning with clickers: Are clickers good for students. Interdisciplinary Journal of E-learning and Learning Objects, 7, 169-184. Bojinova, E. D., & Oigara, J. N. (2013). Teaching and learning with clickers in higher education. International Journal of Teaching and Learning in Higher Education, 25(2), 154-165. Bousbahi, F. (2014). Use of i-clickers to enhance learning outcomes assessment in classroom: A case study in King Saud University. International Journal of Teaching and Education, 2(4), 17-25. Calma, A., Webster, B., Petry, S., & Pesina, J. (2014). Improving the quality of student experience in large lectures using quick polls. Australian Journal of Adult Learning, 54(1), 114-136. Castillo-Manzano, J. I., Castro-Nuño, M., López-Valpuesta, L., Sanz-Díaz, M. T., & Yñiguez, R. (2016). Measuring the effect of ARS on academic performance: A global meta-analysis. Computers & Education, 96, 109-121. https://doi.org/10.1016/j.compedu.2016.02.007 https://doi.org/10.1016/j.chb.2015.11.033 https://doi.org/10.1016/j.compedu.2009.09.033 https://doi.org/10.1016/j.compedu.2012.10.019 https://doi.org/10.1016/j.compedu.2016.02.007 Australasian Journal of Educational Technology, 2019, 35(1). 97 Cavus, N., & Ibrahim, D. (2009). M-learning: An experiment in using SMS to support learning new English language words. British Journal of Educational Technology, 40(1), 78-91. https://doi.org/10.1111/j.1467- 8535.2007.00801 Chanchary, F. H., & Islam, S. A. (2011). Mobile learning in Saudi Arabia–prospects and challenges. Working Paper. Department of Computer Science, Najran University, Najran, Saudi Arabia. Chien, Y. T., Chang, Y. H., & Chang, C. Y. (2016). Do we click in the right way? A meta-analytic review of clicker-integrated instruction. Educational Research Review, 17, 1-18. https://doi.org/10.1016/j.edurev.2015.10.003 Chuang, Y. T. (2015). SSCLS: A smartphone-supported collaborative learning system. Telematics and Informatics, 32(3), 463-474. https://doi.org/0.1016/j.tele.2014.10.004 Churchill, G. A. (1979). A paradigm for developing better measures of marketing constructs. Journal of Marketing Research, 16, 64–73. https://doi.org/10.2307/3150876 Communications and Information Technology Commission. (2014). Number of internet users in Saudi Arabia. Retrieved from http://www.citc.gov.sa/en/Reportsandstudies/Indicators/Pages/CITCICTIndicators.aspx Cotten, S. R., Anderson, W. A., & Tufekci, Z. (2009). Old wine in a new technology or a new type of digital divide? New Media & Society, 11(7), 1163-1186. Cotten, S. R., Shank, D. B., & Anderson, W. A. (2014). Gender, technology use and ownership, and media- based multitasking among middle school students. Computers in Human Behavior, 35, 99-106. Duncan, D. K., Hoekstra, A. R., & Wilcox, B. R. (2012). Digital devices, distraction, and student performance: Does in-class cell phone use reduce learning. Astronomy Education Review, 11(1), 1-4. https://doi.org/10.3847/AER2012011 Escobar-Rodriguez, T., & Monge-Lozano, P. (2012). The acceptance of Moodle technology by business administration students. Computers & Education, 58(4), 1085–1093. https://doi.org/10.1016/j.compedu.2011.11.012 Falk, R. F., & Miller, N. (1992). A primer for soft modelling. Akron, OH: University of Akron Press. Gikas, J., & Grant, M. M. (2013). Mobile computing devices in higher education: Student perspectives on learning with cellphones, smartphones & social media. The Internet and Higher Education, 19, 18-26. https://doi.org/10.1016/j.iheduc.2013.06.002 Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2006). Multivariate data analysis (6th ed.). Upper Saddle River, NJ: Prentice Hall. Hargittai, E., & Walejko, G. (2008). the participation divide: Content creation and sharing in the digital age. Information, Communication, and Society, 11(2), 239-256. Hofstede, G. (1980). Culture‘s consequences: International differences in work-related values. Beverly Hills, CA: Sage Publications. Hunsu, N. J., Adesope, O., & Bayly, D. J. (2016). A meta-analysis of the effects of audience response systems (clicker-based technologies) on cognition and affect. Computers & Education, 94, 102-119. https://doi.org/10.1016/j.compedu.2015.11.013 Hwang, I., Wong, K., Lam, S. L., & Lam, P. (2015). Student response (clicker) systems: Preferences of biomedical physiology students in Asian classes. Electronic Journal of e-Learning, 13(5), 319-330. Irfan, M. (2013). Kingdom tops world in education spending. Retrieved from http://www.arabnews.com/saudi-arabia/kingdom-tops-world-education-spending Johnson, B. & Christensen, L. (2013). Educational research: Quantitative, qualitative and mixed approaches (5th ed.). Los Angeles, CA: Sage. Junco, R., Merson, D., & Salter, D. W. (2010). The effect of gender, ethnicity, and income on college students' use of communication technologies. Cyberpsychology, Behavior, and Social Networking, 13(6), 619-627. https://doi.org/10.1089/cyber.2009.0357 Kahu, E. R. (2013). Framing student engagement in higher education. Studies in Higher Education, 38(5), 758-773. https://doi.org/10.1080/03075079.2011.598505 Keogh, P., & Wang, Z. (2010). Clickers in instruction: One campus, multiple perspectives. Library Hi Tech, 28(1), 8-21. Kim, D., Rueckert, D., Kim, D.-J., & Seo, D. (2013). Students’ perceptions and experiences of mobile learning. Language Learning & Technology, 17(3), 52-73. https://doi.org/10.1111/j.1467-8535.2007.00801 https://doi.org/10.1111/j.1467-8535.2007.00801 https://doi.org/10.1016/j.edurev.2015.10.003 https://doi.org/10.2307/3150876 http://www.citc.gov.sa/en/Reportsandstudies/Indicators/Pages/CITCICTIndicators.aspx https://doi.org/10.3847/AER2012011 https://doi.org/10.1016/j.compedu.2011.11.012 https://doi.org/10.1016/j.iheduc.2013.06.002 https://doi.org/10.1016/j.compedu.2015.11.013 http://www.arabnews.com/saudi-arabia/kingdom-tops-world-education-spending https://doi.org/10.1089/cyber.2009.0357 https://doi.org/10.1080/03075079.2011.598505 Australasian Journal of Educational Technology, 2019, 35(1). 98 Kukulska-Hulme, A. (2009). Will mobile learning change language learning? European Association for Computer Assisted Language Learning, 21(2), 157-165. Kuznekoff, J. H., Munz, S., & Titsworth, S. (2015). Mobile phones in the classroom: Examining the effects of texting, Twitter, and message content on student learning. Communication Education, 64(3), 344-365. Lepp, A., Barkley, J. E., & Karpinski, A. C. (2015). The relationship between cell phone use and academic performance in a sample of US college students. SAGE Open, 5(1), 1-15. https://doi.org/10.1177/2158244015573169 Lopez, J. A., Love, C., & Watters, D. (2014). Clickers in biosciences: Do they improve academic performance? International Journal of Innovation in Science and Mathematics Education (formerly CAL- laborate International), 22(3), 26-41. Margaryan, A., Littlejohn, A., & Vojt, G. (2011). Are digital natives a myth or reality? University students’ use of digital technologies. Computers & education, 56(2), 429-440. Morrell, L. J., & Joyce, D. A. (2015). Interactive lectures: Clickers or personal devices? F1000Research, 4. https://doi.org/ 10.12688/f1000research.6207.1 Nihalani, P. K., & Mayrath, M. C. (2010). Statistics I. Findings from using an iPhone app in a higher education course. White Paper. Retrieved from http://gylo.com/WhitePaper_03302010_Stats1.pdf Parcha, J. M. (2014). Accommodating twitter: Communication accommodation theory and classroom interactions. Communication Teacher, 28(4), 229-235. Pistilli, N., & Cain, J. (2016). Using a health care practice framework to address smartphone use in the classroom. Currents in Pharmacy Teaching and Learning, 8(2), 247-253. https://doi.org/10.1016/j.cptl.2015.12.020 Rashid, T., & Asghar, H. M. (2016). Technology use, self-directed learning, student engagement and academic performance: Examining the interrelations. Computers in Human Behavior, 63, 604-612. https://doi.org/10.1016/j.chb.2016.05.084 Richardson, A. M., Dunn, P. K., McDonald, C., & Oprescu, F. (2015). Crisp: An instrument for assessing student perceptions of classroom response systems. Journal of Science Education and Technology, 24(4), 432-447. https://doi.org/10.1007/s10956-014-9528-2 Rodriguez. J. E. (2011). Social media use in higher education: Key areas to consider for educators. Journal of Online Learning and Teaching, 7(4), 539-550. Sana, F., Weston, T., & Cepeda, N. J. (2013). Laptop multitasking hinders classroom learning for both users and nearby peers. Computers & Education, 62, 24–31. Simpson, V., & Oliver, M. (2007). Electronic voting systems for lectures then and now: a comparison of research and practice. Australasian Journal of Educational Technology, 23(2), 187-208. https://doi.org/10.14742/ajet.1264 Tai, Y. (2012). Contextualizing a MALL: Practice design and evaluation. Educational Technology & Society, 15(2), 220-230. Tse, T., Tiong, J., & Kangaslahti, V. (2004). The effect of cultural norms on the uptake of information and communication technologies in Europe: A conceptual analysis. International Journal of Management 21(3), 382-392. Tweed, R. G., & Lehman, D. R. (2002). Learning considered within a cultural context: Confucian and Socratic approaches. American Psychologist 57(2), 89-99. Vana, K. D., Silva, G. E., Muzyka, D., & Hirani, L. M. (2011). Effectiveness of an audience response system in teaching pharmacology to baccalaureate nursing students. Computers. Nursing Informatics, 29(6), 326- 334. https://doi.org/10.1097/NCN.0b013e3182285d71 Van Raaij, E. M., & Schepers, J. J. (2008). The acceptance and use of a virtual learning environment in China. Computers & Education, 50(3), 838-852. Werts, C. E., Linn, R. L., & Jöreskog, K. G. (1974). Intraclass reliability estimates: Testing structural assumptions. Educational and Psychological Measurement, 34(1), 25–33. https://doi.org/10.1177/001316447403400104 Zaiţ, A., & Bertea, P. S. P. E. (2011). Methods for testing discriminant validity. Management & Marketing Journal, 9(2), 217-224. Zhu, E. (2008). ‘Teaching with clickers’. Occasional Paper No. 22. Center for Research on Learning and Teaching: University of Michigan. Retrieved from https://tccl.arcc.albany.edu/knilt/images/f/fe/Zhu_Teaching_with_Clickers.pdf. https://doi.org/10.1177/2158244015573169 https://doi.org/%2010.12688/f1000research.6207.1 http://gylo.com/WhitePaper_03302010_Stats1.pdf https://doi.org/10.1016/j.cptl.2015.12.020 https://doi.org/10.1016/j.chb.2016.05.084 https://doi.org/10.1007/s10956-014-9528-2 https://doi.org/10.14742/ajet.1264 https://doi.org/10.1097/NCN.0b013e3182285d71 https://doi.org/10.1177/001316447403400104 https://tccl.arcc.albany.edu/knilt/images/f/fe/Zhu_Teaching_with_Clickers.pdf. Australasian Journal of Educational Technology, 2019, 35(1). 99 Corresponding author: Abdulaziz Aljaloud, aaljalou@myune.edu.au Australasian Journal of Educational Technology © 2019. Please cite as: Aljaloud, A., Gromik, N., Kwan, P., & Billingsley, W. (2019). Saudi undergraduate students’ perceptions of the use of smartphone clicker apps on learning performance Australasian Journal of Educational Technology, 35(1), 85-99. https://doi.org/10.14742/ajet.3340 https://doi.org/10.14742/ajet.3340 Introduction Smartphone Traditional clicker Overview Benefits of clicker-based learning Challenges associated with clicker-based learning Research focus Context Conceptual framework, research question, and hypotheses Methodology Procedure Data collection Analysis and results Confirmatory factor analysis Structural model Discussion Implications for practice Limitations Conclusion References