(page number not for citation purpose) 1 *Corresponding author. Email: damar@ppi.ac.id Research in Learning Technology 2023. © 2023 K. J. Anggoro and D. I. Pratiwi. Research in Learning Technology is the journal of the Association for Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. http://www.alt.ac.uk/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license. Citation: Research in Learning Technology 2023, 31: 2944 - http://dx.doi.org/10.25304/rlt.v31.2944 Research in Learning Technology Vol. 31, 2023 ORIGINAL RESEARCH ARTICLE University students’ perceptions of interactive response system in an English language course: a case of Pear Deck Kiki Juli Anggoroa and Damar Isti Pratiwib* aSchool of Languages and General Education, Walailak University, Nakhon Si Thammarat, Thailand; bRailway Mechanical Technology Study Program, Politeknik Perkeretaapian Indonesia Madiun, Madiun, Indonesia (Received: 3 November 2022; Revised: 4 January 2023; Accepted: 2 February 2023; Published: 24 April 2023) Pear Deck is one interactive response system that has gained popularity in recent years. This study addressed the gap in the literature and considered students’ experience of the platform in a Thai university context. This was a mixed-method study in which 320 students completed a survey including closed and open-ended components. Quantitative data measuring students’ perceptions using Likert-scale surveys were collected, while qualitative data were used to get a deeper under- standing of students’ experience in learning using Pear Deck in the classroom. The data were analysed based on gender differences and students’ proficiency levels. According to the findings of this study, students’ perceptions were not significantly different based on gender. However, despite the finding that both basic and inde- pendent users had good attitudes towards the platforms, the latter group gave a substantially higher score. Furthermore, the study revealed that the students had a favourable impression of Pear Deck. They believed that the platform was engag- ing, easy to use, and had the potential to aid learning. Keywords: English language courses; interactive response system; learning technology; Pear Deck; students’ perceptions. Introduction Technology integration has become commonplace in English language teaching around the world. It has been demonstrated that technology-mediated training pro- motes active learning. Active learning is the polar opposite of traditional learning in that it refers to a series of learning activities that result in active engagement and dis- coveries by students (Javed and Odhabi 2018; Jones and Palmer 2017). Active learning in English teaching can take place individually, such as via brainstorming, reading, and writing, or in groups, such as through debates and roleplaying (Jones and Palmer 2017; Kariadi and Pratiwi 2022). Positive learning outcomes have been linked to active learning (El Shaban 2017; Kariadi and Pratiwi 2022; Pratiwi and Waluyo 2022). Active learning encourages student-centred education, which helps pupils build moti- vation, positive perspective, and autonomy in language acquisition (Lei and Lin 2022; Leong et al. 2018). mailto:damar@ppi.ac.id http://www.alt.ac.uk/ http://creativecommons.org/licenses/by/4.0/ http://dx.doi.org/10.25304/rlt.v31.2944 K. J. Anggoro and D. I. Pratiwi 2 Citation: Research in Learning Technology 2023, 31: 2944 - http://dx.doi.org/10.25304/rlt.v31.2944 (page number not for citation purpose) At present, active learning has been improved by using interactive response sys- tems (IRSs) such as Pear Deck, Kahoot, and Quizizz, which provide several advan- tages. IRSs have been recognised for encouraging student participation in class and assisting students in directing their attention, offering real-time feedback, and assess- ing students’ understanding (Collins 2007; Paul et al. 2020). Waluyo (2020) found that students’ success across all skills was enhanced by incorporating active learning and smart classroom tools such as Socrative, Google Forms, and Facebook into basic English courses. Since its debut in 2014, Pear Deck has grown in popularity among a wide range of users. This platform enables students to give real-time responses in various formats, including selecting from multiple choices, writing short and long pas- sages and drawing. Recent studies reported that Pear Deck could enhance students’ engagement in class (Dong et al. 2018). Most importantly, Pear Deck can improve students’ achievement (Liu, Sands-Meyer, and Audran 2019). However, little research on the platform in English as a Foreign Language (EFL) contexts in Asia, including Thailand, has been conducted. It is critical to comprehend the experiences of its users to develop future uses of the platform (Anggoro 2021). As a result, the focus of this study was on Thai university students’ perceptions of Pear Deck in English language courses. This study also compared students’ perceptions based on gender and English proficiency levels. Pear Deck as an IRS An IRS, as defined by Liu (2003), is a technology-enabled learning environment that improves student engagement. An IRS, as defined by Awedh et al. (2014), is an online student response system that allows teachers to test student comprehension and track their progress by assigning educational assignments. An IRS enables educators to improve student participation in the classroom, stimulate deeper conversation, foster cooperation, and provide quick feedback (Turner 2015). An IRS is typically com- prised of hardware, such as a set of simple personal handheld signal transmitters and a response signal receiver connected to a classroom computer to collect responses from classroom participants, and software configured on a classroom computer to process the collected responses and present the results on a screen (Liu 2003). None- theless, as technology advances, the IRS increasingly employs mobile technologies, which are practical for students. Recent IRS research included the usage of students’ mobile phones (Balta and Hamza 2017; Sun and Hsieh 2018). Pear Deck, a platform that combines slides with interactive features, is one IRS that has gained much popularity. Mache et al. (2017) mentioned that Pear Deck acts as an online presenting platform that is combined with an interactive classroom response system. This platform has a real-time response system that enables teach- ers to observe students’ answers synchronously; thus, giving immediate feedback becomes possible (Anggoro 2021). Concerning accessibility, Liu, Sands-Meyer and Audran (2019) stated that Pear Deck is more convenient, simple to use, and inex- pensive than other student response systems because any digital device with internet connectivity can access the platform. Anggoro (2021) also states that people famil- iar with Google Slides and Microsoft PowerPoint can quickly learn how to operate Pear Deck. On the platform, students can promptly respond to the various types of ques- tions provided in the interactive slide, and teachers can check the students’ answers http://dx.doi.org/10.25304/rlt.v31.2944 Research in Learning Technology Citation: Research in Learning Technology 2023, 31: 2944 - http://dx.doi.org/10.25304/rlt.v31.2944 3 (page number not for citation purpose) simultaneously through the teacher dashboard (Sengsri and Anggoro 2021). Teachers can display the answers anonymously on the projector screen while giving verbal feed- back (Dong et al. 2018). Pear Deck also enables review sessions. Teachers can access all students’ answers any time after the lesson (Anggoro and Khasanah 2022). At the same time, students receive takeaways which are their answers, and the slides are sent back to them through email (Anggoro 2021). Several studies reported on the benefits of Pear Deck to learning. Ni et al. (2020) studied the use of the site along with a game to facilitate vocabulary instruction, and their investigation found that students’ motivation and achievement were improved. Mache et al. (2017) reported that the tool could improve students’ engagement. This study is congruent with Javed and Odhabi (2018), who found that the platform can improve engagement and facilitate students’ active learning. In EFL, Pear Deck was also utilised in several studies. Liu, Sands-Meyer and Audran (2019) utilised Pear Deck to facilitate English grammar learning and reported improved learning moti- vation and self-efficacy in learning English grammar. In addition, to use it in a class- room, Pear Deck can also enhance online instruction in terms of solving problems with students’ engagement and participation (Anggoro 2021). Since previous research (Archambault, Pagani, and Fitzpatrick 2013; Javed and Odhabi 2018) reported that engagement relates to achievement, Pear Deck has the potential to improve online learning. Students’ perceptions of technology tools based on gender and English proficiency levels To investigate students’ perceptions of Pear Deck, further, this study aims to analyse students’ comments based on their gender. Even though technology has been widely employed, gender differences should be considered. According to several research- ers, female students are more comfortable utilising technology tools than male pupils (Pratiwi and Ubaedillah 2021; Shin et al. 2018). Their studies revealed that women were more likely to believe that technology tools were useful. These findings con- tradicted several other studies (Cuadrado-García, Ruiz-Molina, and Montoro-Pons 2010; Tucker 2014). Tucker (2014) found that men were more involved in discussions utilising computers as new media, as a result they gained more helpful knowledge than women who perceived computers as more of a social media tool. Furthermore, Cuadrado-García, Ruiz-Molina and Montoro-Pons (2010) discovered that young men are more familiar with virtual language than women, influencing their online performance. Another finding showed no substantial differences between genders (Gunamala and Sneha 2013). They stated that gender disparities had little effect on their ability to use technology tools and both sexes had similar attitudes towards it. Another discussion is on the connection between students’ perceptions of Pear Deck and their English proficiency levels. Previous studies have disclosed the benefits of Pear Deck for English learners. The platform can be advantageous for improving students’ engagement (Mache et al. 2017) and achievement (Anggoro and Khasanah 2022). Also, it is user-friendly (Liu, Sands-Meyer, and Audran 2019). Nonetheless, there is a lack of studies discussing how groups of learners with different English pro- ficiency levels react to Pear Deck. By knowing the reactions of learners from different English proficiency levels, we might figure out which group might benefit more from its utilisation. http://dx.doi.org/10.25304/rlt.v31.2944 K. J. Anggoro and D. I. Pratiwi 4 Citation: Research in Learning Technology 2023, 31: 2944 - http://dx.doi.org/10.25304/rlt.v31.2944 (page number not for citation purpose) Methodology Research objectives and design This research aimed to investigate students’ perceptions of using Pear Deck in English language classes. Furthermore, it compared students’ perceptions of Pear Deck in relation to their genders and English proficiency levels. The mixed-method research design was utilised to achieve the research objectives. It was used to strengthen the triangulation of data (Creamer 2017). Triangulation of data helps to achieve trust- worthiness (Korstjens and Moser 2018). Hence, it is expected that the results obtained in this study are trustworthy. Setting and participants The study was conducted at a university in the south of Thailand. A total of 320 con- sented students participated in the survey. The students, aged from 19 to 21 years old, were enrolled in a course named English communication skills. The students came from eight different sections. Each section approximately consisted of 35–40 stu- dents. They were diverse in relation to their English proficiency levels. To determine their English proficiency levels, their Common European Framework of Reference for Languages (CEFR) levels were investigated from the results of their university English proficiency tests. CEFR has been widely used worldwide and has impacted language standards, curricula, and revision reform (North 2014). The most prominent part of the framework is its vertical dimension, which includes proficiency levels rang- ing from basic user to proficient user (Huhta 2012). Most importantly, the participants were taught using Pear Deck slides. Five classes were onsite and seven were online because of the re-rising case of Covid-19 (COVID- 19). Pear Deck was often utilised from the beginning to the end of the class for lesson review and language practice purposes. Two lecturers designed Pear Deck slides for 2-h classes. The slides normally included several activities, including a warm-up activity, vocabulary building, reading or listening activity, grammar points, and speaking or writing activity. The following Figure 1 shows the types of slides used in each activity: Data collection An online questionnaire consisting of closed-ended and open-ended sections was uti- lised to collect data. A five-point Likert scale was used in the closed section. The ques- tions were about the use of IRS in online English classes. For instance, students were given a statement, ‘I understand the lesson more because of Pear Deck’., and asked to Figure 1. Flow of lesson and types of Pear Deck slides used. http://dx.doi.org/10.25304/rlt.v31.2944 Research in Learning Technology Citation: Research in Learning Technology 2023, 31: 2944 - http://dx.doi.org/10.25304/rlt.v31.2944 5 (page number not for citation purpose) select an option that came closest to their opinion. The options were Strongly Agree (SA), Agree (A), Undecided (U), Disagree (D), or Strongly Disagree (SD). More sam- ple statements are available in the following Table 1. In the open-ended section, stu- dents had the opportunity to write comments and opinions on their experiences with Pear Deck as well as suggestions for future use. Data analysis The gathered data were held in strict confidence and only accessible to the researchers. Throughout the study’s duration, participants were informed of and assured of their confidentiality. The collected quantitative data were analysed by using descriptive sta- tistics and inferential statistics. Content analysis was applied to analyse the qualita- tive data. The collected qualitative data were carefully and repetitively examined to discover themes, which were then classified and organised according to the research questions given for this study. Results Students’ perception of Pear Deck in an English language class Table 2 shows that students’ overall perception of Pear Deck in an English language class is high (M = 4.32, SD = 0.70). Table 3 further explains students’ perceptions based on three categories, engagement, accessibility, and achievement. The mean in each category is over 4.0, showing the students’ positive perceptions. In the open-ended section of the survey, 85% of the students left a comment, while 15% did not write anything. From 85% of the comments, 81% showed a positive reaction to the use of Pear Deck in class. A total of 4% students conveyed about some problems they had with the platform. Students’ positive comments were categorised into three parts: engagement, convenience, and achievement. The following Table 4 contains some students’ comments based on the three categories. Table 1. Sample items from the survey. Perceptions Item samples Achievement 1. I understand the lesson more because of Pear Deck. 2. Activities on Pear Deck make me practice and understand more. Engagement 1. I feel engaged when Pear Deck is used. 2. Pear Deck makes my class more fun. Convenience 1. Pear Deck is convenient to use. 2. I am happy to use Pear Deck in my classes. Table 2. Descriptive statistics of students’ overall perception. Descriptive statistics N Minimum Maximum Mean Std. deviation Perception 320 1.00 5.00 4.3275 0.70701 Valid N (listwise) 320 http://dx.doi.org/10.25304/rlt.v31.2944 K. J. Anggoro and D. I. Pratiwi 6 Citation: Research in Learning Technology 2023, 31: 2944 - http://dx.doi.org/10.25304/rlt.v31.2944 (page number not for citation purpose) As previously mentioned, several students mentioned a major issue they had while using Pear Deck in an online class. The issue is a bad internet signal. The following are students’ comments. Yes, Pear Deck is difficult to use because the internet is lost, and the images in the Webex are not exactly in the Pear Deck. Yes, but I have a problem with the internet. I think it is good. But stuck with the problem that the internet contract is unstable. Students’ perceptions based on gender To gain more in-depth results, students’ perceptions were compared based on their gender. Table 5 points out that both female and male students showed a positive atti- tude towards the use of Pear Deck. Additionally, the independent samples t-test, as shown in Table 6, discloses that there was no significant difference between the female (M = 4.36, SD = 0.68) and male (M = 4.20, SD = 0.76) students’ perceptions, t(318) = 1.65, p = 0.41. However, the mean of female students’ perception was slightly higher than that of the male. Students’ perceptions in the open-ended section were also categorised based on their gender. The majority of both female and male students showed positive per- ceptions of Pear Deck. The comments from both genders can also be categorised Table 3. Descriptive statistics of students’ perception by categories. No. Categories Mean SD 1 Engagement 4.36 0.72 2 Convenience 4.32 0.72 3 Achievement 4.29 0.75 Table 4. Students’ positive comments. No. Categories Comments 1 Engagement 1.1. Can make us excited and fun in learning. 1.2. Using Pear Deck is a lot of fun and made the lesson more understandable. 1.3. It is good for using the app this year, study online because it can create more enthusiasm for students to study online. 2 Convenience 2.1. Easy to get into, easy to use. 2.2. Nice because the teacher can see the student’s answer so I have no need to say it again and again due to my poor inter- net connection. 2.3. It is convenient, fast and easy to access. 3 Achievement 3.1. Activities on the Pear Deck made me practice and under- stand more, make learning more efficient 3.2. The teacher taught me vocabulary, grammar through this programme and made me understand. 3.3. It is a good app for learning English skills, including writing, listening, and drawing. http://dx.doi.org/10.25304/rlt.v31.2944 Research in Learning Technology Citation: Research in Learning Technology 2023, 31: 2944 - http://dx.doi.org/10.25304/rlt.v31.2944 7 (page number not for citation purpose) into engagement, convenience, and achievement. The minority (4%) who wrote about internet issues were all female students. Students’ perceptions based on English proficiency levels Students’ proficiency levels were categorised into two parts: the basic user or CEFR A1 and A2 and the independent user or CEFR B1 and B2. Table 7 discloses that the mean of the independent users’ perception (m = 4.44) is higher than that of the basic user (m = 4.20). An independent samples t-test, as shown in Table 8, further elaborates that there was a significant difference in the perceptions between the inde- pendent users (M = 4.44, SD = 0.53) and basic users (M = 4.20, SD = 0.84), t(318) = 2.98, p = 0.00. Students’ perceptions in the open-ended section were also categorised based on their English proficiency levels: basic user and independent user. The majority of both levels showed positive perceptions of Pear Deck. The comments can also be catego- rised into engagement, convenience, and achievement. The minority (4%) who wrote about internet issues came from two different levels: 2.5% were from the basic user group and 1.5% were independent users. Discussion The use of IRSs has become more common at present. Pear Deck is one platform that can be utilised in English classes. This study reports that the platform was enjoyed by the vast majority of survey respondents. Most respondents agreed or strongly agreed with positive statements about Pear Deck in the survey. The open-ended section of the survey also mainly consisted of favourable comments on the platform. One prom- inent category in the findings is engagement. The findings show that Pear Deck has the potential to engage English learners during class. This is in line with the finding of Archambault, Pagani and Fitzpatrick (2013) and Mache et al. (2017). Students also conveyed that they became more active in class when Pear Deck was used. This is congruent with the finding of Javed and Odhabi (2018) that Pear Deck can facilitate active learning. Another category is convenience or accessibility. This category is important for IRSs because they are expected to make the class easier, not more complicated. A large number of the survey respondents agreed that using Pear Deck was convenient. It was user friendly and did not require intensive training. This finding supports that of Anggoro (2021) about the easy accessibility of Pear Deck. It is also congruent with the finding of Liu, Sands-Meyer and Audran (2019). However, for students with a bad internet signal, the experience might be different. The minority of the respon- dents informed us that due to unreliable internet connection, they were not able to Table 5. Descriptive statistics of students’ perception based on gender. Group statistics Gender N Mean Std. deviation Std. error mean Perception Female 250 4.3620 0.68685 0.04344 Male 70 4.2043 0.76735 0.09172 http://dx.doi.org/10.25304/rlt.v31.2944 K. J. Anggoro and D. I. Pratiwi 8 Citation: Research in Learning Technology 2023, 31: 2944 - http://dx.doi.org/10.25304/rlt.v31.2944 (page number not for citation purpose) T ab le 6 . In d ep en d en t sa m p le s t- te st f o r st u d en ts ’ p er ce p ti o n b as ed o n g en d er . In d ep en d en t sa m p le s te st L ev en e’ s te st f o r eq u al it y o f va ri an ce s t- te st f o r eq u al it y o f m ea n s F S ig . t d f S ig . (2 -t ai le d ) M ea n D if fe re n ce S td . E rr o r D if fe re n ce 95 % C o n fi d en ce I n te rv al o f th e D if fe re n ce L o w er U p p er P er ce p ti o n E q u al va ri an ce s as su m ed 0. 67 1 0. 41 3 1. 65 4 31 8 0. 09 9 0. 15 77 1 0. 09 53 5 –0 .0 29 88 0. 34 53 0 E q u al va ri an ce s n o t  as su m ed 1. 55 4 10 2. 00 8 0. 12 3 0. 15 77 1 0. 10 14 8 –0 .0 43 58 0. 35 90 1 http://dx.doi.org/10.25304/rlt.v31.2944 Research in Learning Technology Citation: Research in Learning Technology 2023, 31: 2944 - http://dx.doi.org/10.25304/rlt.v31.2944 9 (page number not for citation purpose) access the platform. A similar finding was also reported by Anggoro and Khasanah (2022). Hence, a stable internet signal is a prerequisite to utilising the platform. The last category is achievement. The respondents positively reacted to statements on learning achievement. They mentioned that the platform made them understand the lesson more. Several comments made by students talked about the use of Pear Deck to learn specific English skills, including vocabulary, listening, writing, read- ing, and grammar. Hence, the platform has the potential to help improve students’ English skills. This is in line with the finding of Archambault, Pagani and Fitzpatrick (2013) and Javed and Odhabi (2018). Several students also pointed out that due to the COVID-19 restrictions, they were forced to study online. They further explained that using Pear Deck made them able to comprehend the online class more, and they were able to practice several English skills during class. This finding supports that of Anggoro and Khasanah (2022) about the use of Pear Deck in an online class. Apart from the three categories, this study discussed students’ perceptions based on gender and English proficiency levels. Based on gender, even though the female participants did score a higher mean than the male, there was no significant differ- ence in the respondents’ perceptions of Pear Deck. Hence, the finding that female students are more comfortable using technological tools than male might not be in line with that of Huo (2013) and Shin et al. (2018). This finding is more congruent to that of Gunamala and Sneha (2013), that there are no significant differences between the female and male students’ performances in using technology tools for learning. In relation to the English proficiency level, this study found that both the basic user and independent user groups showed positive reactions to Pear Deck. Both groups conveyed their satisfaction with the use of the platform and wished that it would be used in other courses as well. The independent user groups, moreover, scored a significantly higher mean than that of the basic user. This implies that Pear Deck might be a beneficial platform to select to teach basic user students and it might be more advantageous for independent user students. The basic user students can enjoy the lesson more, while the independent user students can increase their achievement. Conclusion As an IRS, Pear Deck has been used in classrooms, especially in English class of higher education context. This study reports that the platform might have the poten- tial to improve English classes by engaging students and assisting their learning. The platform is also easy to use, so that students will get used to it quickly. Pear Deck is a versatile tool since it can be used for both onsite and online classes as long as the internet connection is strong. Students with bad internet signals might struggle to participate in activities on the platform. This study also discussed students’ percep- tions based on gender and English proficiency levels and found that the platform was Table 7. Descriptive statistics of students’ perception based on English proficiency levels. Group statistics English proficiency (EP) N Mean Std. deviation Std. error mean Perception Basic user 155 4.2071 0.84000 0.06747 Independent user 165 4.4406 0.53213 0.04143 http://dx.doi.org/10.25304/rlt.v31.2944 K. J. Anggoro and D. I. Pratiwi 10 Citation: Research in Learning Technology 2023, 31: 2944 - http://dx.doi.org/10.25304/rlt.v31.2944 (page number not for citation purpose) T ab le 8 . In d ep en d en t sa m p le s t- te st f o r st u d en ts ’ p er ce p ti o n s b as ed o n E n gl is h p ro fi ci en cy l ev el s. In d ep en d en t sa m p le s te st L ev en e’ s te st f o r eq u al it y o f va ri an ce s t- te st f o r eq u al it y o f m ea n s F S ig . t d f S ig . (2 -t ai le d ) M ea n d if fe re n ce S td . er ro r d if fe re n ce 95 % c o n fi d en ce i n te rv al o f th e d if fe re n ce L o w er U p p er P er ce p ti o n E q u al v ar ia n ce s as su m ed 12 .8 35 0. 00 0 -2 .9 89 31 8 0. 00 3 -0 .2 33 51 0. 07 81 2 -0 .3 87 21 -0 .0 79 81 E q u al v ar ia n ce s n o t as su m ed -2 .9 49 25 7. 61 7 0. 00 3 -0 .2 33 51 0. 07 91 7 -0 .3 89 42 -0 .0 77 60 http://dx.doi.org/10.25304/rlt.v31.2944 Research in Learning Technology Citation: Research in Learning Technology 2023, 31: 2944 - http://dx.doi.org/10.25304/rlt.v31.2944 11 (page number not for citation purpose) enjoyed by both female and male students, as well as the two different English profi- ciency groups found in the study: basic users and independent users. Although both users showed a positive attitude towards the use of Pear Deck, the independent user showed higher satisfaction than the basic user. There are several limitations to this study. Firstly, the imbalance of the number of students based on gender. The female and male students were 250 and 70 people, respectively. Secondly, it was the English proficiency levels. There were only two out of three levels found. No advanced user participated in the study. The last was the dif- ferent environments where Pear Deck was utilised. Several students had experiences using it in the classroom, while others were online. A future study comparing students’ perceptions of the platform in online, onsite, and hybrid environments might enrich the results of this study. References Anggoro, K. J. (2021) Pear deck. RELC Journal, vol. 52, no. 3, pp. 645–647. doi: 10.1177/0033688220936735 Anggoro, K. J. & Khasanah, U. (2022) EFL students’ independent learning with pear deck interactive slides. Studies in Self-Access Learning Journal, vol. 13, no. 1, pp. 172–176. doi: 10.37237/130111 Archambault, I., Pagani, L. S. & Fitzpatrick, C. (2013) Transactional associations between classroom engagement and relations with teachers from first through fourth grade. Learning and Instruction, vol. 23, pp. 1–9. doi: 10.1016/j.learninstruc.2012.09.003 Awedh, M., et al., (2014) Using socrative and smartphones for the support of collaborative learning. International Journal on Integrating Technology in Education, vol. 3, no. 4, pp. 17–24. doi: 10.5121/ijite.2014.3402 Balta, N. & Hamza, M. (2017) The effect of student collaboration in solving physics problems using an online interactive response system. European Journal of Educational Research, vol. 6, no. 3, pp. 385–394. doi: 10.12973/eu-jer.6.3.385 Collins, L. J. (2007) Livening up the classroom. Medical Reference Services Quarterly, vol. 26, no. 1, pp. 81–88. doi: 10.1300/J115v26n01\_08 Creamer, E. G. (2017) An introduction to fully integrated mixed methods research, 1st edn, Los Angeles: Sage. Cuadrado-García, M., Ruiz-Molina, M.-E. & Montoro-Pons, J. D. (2010) Are there gender dif- ferences in e-learning use and assessment? evidence from an interuniversity online project in Europe. Procedia – Social and Behavioral Sciences, vol. 2, no. 2, pp. 367–371. doi: 10.1016/j. sbspro.2010.03.027 Dong, Y., et al., (2018) Interactive presentation tools using mobile devices. Society for Information Technology & Teacher Education International Conference, vol. 2018, no. 1, pp. 743–748. Available at: http://www.learntechlib.org/p/182605/ El Shaban, A. (2017) The use of socrative in ESL classrooms. Teaching English with Technology, vol. 17, no. 4, pp. 64–77. Available at: http://www.tewtjournal.org Gunamala, S. & Sneha, S. (2013) Application of a gamified interactive response system to enhance the intrinsic and extrinsic motivation, student engagement, and attention of English learn- ers. International Journal of Engineering Research, vol. 2, no. 2. Available at: https://www. indianjournals.com/ijor.aspx?target=ijor:ijer&volume=2&issue=2&article=022 Huhta, A. (2012) Common European framework of reference. In The encyclopedia of applied linguistics. doi: 10.1002/9781405198431.wbeal0157 Wiley-Blackwell: New Jersey. Huo, Q. (2013) Gender Difference in Perception and Adoption of Technology to Enhance Second Language Learning Out-of-School, HKU Theses Online (HKUTO), The University of Hong Kong. http://dx.doi.org/10.25304/rlt.v31.2944 https://doi.org/10.1177/0033688220936735 https://doi.org/10.37237/130111 https://doi.org/10.1016/j.learninstruc.2012.09.003 https://doi.org/10.5121/ijite.2014.3402 https://doi.org/10.12973/eu-jer.6.3.385 https://doi.org/10.1300/J115v26n01 https://doi.org/10.1016/j.sbspro.2010.03.027 https://doi.org/10.1016/j.sbspro.2010.03.027 http://www.learntechlib.org/p/182605/ http://www.tewtjournal.org https://www.indianjournals.com/ijor.aspx?target=ijor https://www.indianjournals.com/ijor.aspx?target=ijor https://doi.org/10.1002/9781405198431.wbeal0157 K. J. Anggoro and D. I. Pratiwi 12 Citation: Research in Learning Technology 2023, 31: 2944 - http://dx.doi.org/10.25304/rlt.v31.2944 (page number not for citation purpose) Javed, Y. & Odhabi, H. (2018) Active learning in classrooms using online tools: evaluating pear- deck for students’ engagement. 2018 Fifth HCT Information Technology Trends (ITT), pp. 126–131. doi: 10.1109/CTIT.2018.8649515 Jones, B. A. & Palmer, R. (2017) Active learning in Japanese university EFL classes: clari- fying the construct. Hirao School of Management Review, vol. 7, pp. 107–125. doi: 10.14990/00002306 Kariadi, M. T. & Pratiwi, D. I. (2022) Revealing university students ’ attitudes toward English language. The Journal of Asia TEFL, vol. 19, no. 3, pp. 1053–1062. doi: 10.18823/ asiatefl.2022.19.3.20.1053 Korstjens, I. & Moser, A. (2018) Series: practical guidance to qualitative research. Part 4: trust- worthiness and publishing. European Journal of General Practice, vol. 24, no. 1, pp. 120– 124. doi: 10.1080/13814788.2017.1375092 Lei, J. & Lin, T. (2022) Emergency online learning: the effects of interactional, motivational, self-regulatory, and situational factors on learning outcomes and continuation intentions. International Review of Research in Open and Distributed Learning, vol. 23, no. 3, pp. 43–60. doi: 10.19173/irrodl.v23i3.6078 Leong, L. C., et al., (2018) Mobile x-space design, teaching strategies and undergraduate students’ collaborative learning behaviour: a case study in Taylor’S university, Malaysia. Malaysian Journal of Learning and Instruction, vol. 15, no. 2, pp. 175–205. doi: 10.32890/mjli2018.15.2.7 Liu, C., Sands-Meyer, S. & Audran, J. (2019) The effectiveness of the student response sys- tem (SRS) in English grammar learning in a flipped English as a foreign language (EFL) class. Interactive Learning Environments, vol. 27, no. 8, pp. 1178–1191. doi: 10.1080/10494820.2018.1528283 Liu, T. C. (2003) The features and potential of interactive response system. Proceedings of International Conference on Computers in Education (ICCE), pp. 315–322 . 2–5 December. Hong Kong. Mache, J., et al., (2017) Pear deck: an interactive classroom response system to encourage stu- dent engagement. Journal of Computing Sciences in Colleges, vol. 33, no. 1, pp. 156–158. Ni, C. K., et al., (2020) Enhancing Malaysian primary pupils’ vocabulary skills using pocable game and pear deck. International Journal of Learning, Teaching and Educational Research, vol. 19, no. 6, pp. 145–160. doi: 10.26803/IJLTER.19.6.9 North, B. (2014) The CEFR in practice. Cambridge University Press, Cambridge. Paul, M. W., et al., (2020) Engaging the control-value theory: a new era of student response systems and formative assessment to improve student achievement. Research in Learning Technology, vol. 28, no. 1063519, pp. 1–16. doi: 10.25304/rlt.v28.2454 Pratiwi, D. I. & Ubaedillah, U. (2021) Digital vocabulary class in english for railway mechan- ical technology. Teaching English with Technology, vol. 21, no. 3, pp. 67–88. Available at: https://tewtjournal.org/download/digital-vocabulary-class-in-english-for-railway-mechani- cal-technology-by-damar-isti-pratiwi-and-ubaedillah-ubaedillah/ Pratiwi, D. I. & Waluyo, B. (2022) Integrating task and game-based learning into an online TOEFL preparatory course during the COVID-19 outbreak at two Indonesian higher edu- cation institutions. Malaysian Journal of Learning & Instruction, vol. 19, no. 2, pp. 37–67. doi: 10.32890/mjli2022.19.2.2 Sengsri, S. & Anggoro, K. J. (2021) Students’ perceptions of content and language integrated pedagogy via electronic communication (E-clip). World Journal on Educational Technology: Current Issues, vol. 13, no. 1, pp. 54–67. doi: 10.18844/wjet.v13i1.5363 Shin, I.-S., et al., (2018) Evaluations of interactive learning tools among engineering students: effects of grit and gender. ASEE IL-IN Section Conference. Available at: http://www.ncbi.nlm.nih.gov/ pubmed/15636205%0Ahttps://docs.lib.purdue.edu/aseeil-insectionconference/2018/assess/3/ Sun, J. C.-Y. & Hsieh, P.-H. (2018) Application of a gamified interactive response system to enhance the intrinsic and extrinsic motivation, student engagement, and attention of English learners. Journal of Educational Technology & Society, vol. 21, no. 3, pp. 104–116. http://www.jstor.org/stable/26458511 http://dx.doi.org/10.25304/rlt.v31.2944 https://doi.org/10.1109/CTIT.2018.8649515 https://doi.org/10.14990/00002306 https://doi.org/10.18823/asiatefl.2022.19.3.20.1053 https://doi.org/10.18823/asiatefl.2022.19.3.20.1053 https://doi.org/10.1080/13814788.2017.1375092 https://doi.org/10.19173/irrodl.v23i3.6078 https://doi.org/10.32890/mjli2018.15.2.7 https://doi.org/10.1080/10494820.2018.1528283 https://doi.org/10.26803/IJLTER.19.6.9 https://doi.org/10.25304/rlt.v28.2454 https://tewtjournal.org/download/digital-vocabulary-class-in-english-for-railway-mechanical-technology-by-damar-isti-pratiwi-and-ubaedillah-ubaedillah/ https://tewtjournal.org/download/digital-vocabulary-class-in-english-for-railway-mechanical-technology-by-damar-isti-pratiwi-and-ubaedillah-ubaedillah/ https://doi.org/10.32890/mjli2022.19.2.2 https://doi.org/10.18844/wjet.v13i1.5363 http://www.ncbi.nlm.nih.gov/pubmed/15636205%0Ahttps http://www.ncbi.nlm.nih.gov/pubmed/15636205%0Ahttps http://docs.lib.purdue.edu/aseeil-insectionconference/2018/assess/3/ http://www.jstor.org/stable/26458511 Research in Learning Technology Citation: Research in Learning Technology 2023, 31: 2944 - http://dx.doi.org/10.25304/rlt.v31.2944 13 (page number not for citation purpose) Tucker, R. (2014) Sex does not matter: gender bias and gender differences in peer assessments of contributions to group work. Assessment & Evaluation in Higher Education, vol. 39, no. 3, pp. 293–309. doi: 10.1080/02602938.2013.830282 Turner, D. (2015) Psychosocial aspects of engagement with social media and digital technol- ogy – personal thoughts from the frontier. In Smart Learning: teaching and learning with smartphones and tablets in post-compulsory education. Media-Enhanced Learning Special Interest Group and Sheffield Hallam University. Available at: http://melsig.shu.ac.uk Waluyo, B. (2020) Learning outcomes of a general English course implementing multiple e-learning technologies and active learning concepts. The Journal of Asia TEFL, vol. 17, no. 1, pp. 160–181. doi: 10.18823/asiatefl.2020.17.1.10.160 http://dx.doi.org/10.25304/rlt.v31.2944 https://doi.org/10.1080/02602938.2013.830282 http://melsig.shu.ac.uk https://doi.org/10.18823/asiatefl.2020.17.1.10.160