Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 Re-engineering challenging and abstract topics using Kahoot!, a student response system Maria Gebbels University of Greenwich Abstract The pervasive nature of digital technologies and students’ habitual use of them brings opportunities for their assimilation into the classroom environment. The ‘student response system’ (SRS), Kahoot!, which allows students instantly to provide answers to multiple- choice questions, was introduced in the Research Methods course. The motivation for using such technology was to reinforce key concepts, often very abstract, consolidate students’ learning, provide them with real-life feedback and help them to grow in their knowledge of, and confidence in engaging with, the subject matter. This case study, based on a small- scale exploratory research sample, showcases the benefits of incorporating this SRS in the Research Methods tutorials. These benefits, as student feedback suggests, include better conceptual understanding and increased motivation, resulting in the transition from surface to deep learning Keywords: student response system; learning technology; consolidation of learning; student engagement; confidence, feedback Introduction This case study is based on the author’s evaluation of the effects and benefits of implementing Kahoot!, also known as a ‘student response system’ (SRS), in the delivery of material for research methods tutorials to second-year undergraduate hospitality management students. The ever-increasing use of digital technologies among students provides opportunities for the enhancement of their learning, as well as their overall engagement with and enjoyment of their chosen university programmes. Because technology expectations (in particular those of young people) are changing, the key aspects of learning – namely listening, seeing and experiencing – can be met and expanded upon by means of Kahoot!. Furthermore, the exponential growth of digital technologies means that, as young people now enter university fairly conversant with them, they can be deployed in a classroom environment for learning and teaching purposes (Lai and Hong, 2015). Since such digital natives, who have grown up surrounded by technology, are likely to learn differently, teachers in Higher Education (HE) should consider accommodating their students’ alternative learning styles with “more technology-driven, spontaneous, and multi- sensory” approaches (Prensky, 2001; Lai and Hong, op.cit., p.726). It is worth noting that the term ‘digital natives’ continues to be debated in the literature in terms of its scope and implications for teaching and learning strategies (Maton and Kervin, 2008; Kivunja, 2014; Šorgo et al., 2017). By incorporating this particular SRS into her teaching, the author wanted students to make productive use of their mobile devices during tutorials and, at the same time, find out if there would be an improvement in how students learn and gain knowledge of an abstract topic (Mayer, 2005). Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 Kahoot! is a free, game-based, learning platform that makes it fun to learn any subject, in any language, on any device and at any age. It is also classified as a SRS, an electronic voting system that enables students to choose answers to multiple-choice questions, inside or outside the classroom environment (Nielsen et al., 2012). In the literature, SRSs are often referred to as ‘classroom response systems’ (CRSs), ‘audience response systems’ (ARSs) or, simply, ‘voting tools’ or ‘clickers’ (Nielsen et al., op.cit; Compton and Allen, 2018, p.1). Kahoot! is one of at least seven different types of technological solution available on the market which are used to widen participation in face-to-face taught sessions. The other types of SRS are Todaysmeet, Slido, Polleverywhere, Mentimeter, Socrative and Zeetings (Compton and Allen, op.cit.). They do have certain characteristics in common, but contrasting capabilities. For instance, Todaysmeet is an online ‘chat room’ that acts as a backchannel to taught sessions. It is simple to set up and use, but messages are limited to 140 characters and it requires paid subscription. Slido offers anonymous polling options in multi-choice, open-text or ranking formats and is effective when used during lectures to gather feedback or enhance student interaction; although it is free, each event is limited to only three polls and lecturers have little control over the content posted by students. Polleverywhere enables students to take part in a poll using a text message, the frequency and content of which can be monitored by a lecturer; although a free version is available, it does not offer any methods of moderation and limits the number of responses. Kahoot! increases classroom engagement by gamifying the sessions with quiz-based activities; students use internet-enabled devices, such as their mobile phones or tablets, to choose one of four shapes corresponding to possible-answer shapes they see on the main classroom screen (Figure 1). Before students can play Kahoot!, all they need to do is to follow a web link and then choose a name for themselves, whether their own or a pseudonym. A comprehensive review of these tools can be found in a technological review by Compton and Allen (op.cit., pp.4-17). Figure 1: Kahoot! main screen example Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 In her own teaching, the author observed that students often struggle to make the transition from surface to deep knowledge when taking the Research Methods course. Also, students often treat the material of the research methods as a recipe to be memorised and not as tools to enable them to conduct independent research in the future. This may be partly owing to the abstract nature of the content covered as well as the short delivery of the material, in term 2 only. Thus, the aim of this case study was to explore the benefits of using Kahoot! in Research Methods tutorials. The main reasons for incorporating this particular SRS in the classroom setting was to reinforce students’ knowledge of the research methods, to enhance their ability to learn, to increase their overall confidence and to improve the classroom atmosphere with a focus on individual interaction followed by peer discussion. This case study is divided into the following sections: a brief introduction to Kahoot!; the reasons for introducing this particular SRS; critical discussion of the existing research on the use of SRS in a classroom environment; description of the sample, implementation and evaluation, with direct quotes from students; and, finally, limitations and conclusions. Literature review Three key themes emerged after conducting a literature review of the use of SRS in a classroom and they are used to guide the following discussion. Benefits of using ‘student response systems’ Existing research revealed the benefits of using SRSs, which are: increased student motivation and engagement; easier clarification of misunderstanding; promotion of active learning; enhanced student performance; better conceptual understanding; instant feedback; the possibility of increasing out-of-class study time; self-assessment of one’s knowledge; a positive classroom atmosphere; engagement and accountability; and increased reinforcement of key concepts (cf. Hedgcock and Rouwenhorst, 2014; Draper and Brown, 2004; Masikunas, Panayiotidis and Burke, 2007; Rice and Bunz 2006; Boyle and Nicol, 2003; Rao and DiCarlo, 2000; Crouch and Mazur, 2001; Nielsen et al., op.cit; Caldwell, 2007). Despite the many benefits, technology by itself, no matter how innovative or engaging, will not make the sessions better (Mayer, op.cit.). It is critical to know the profile of the students and to identify clear objectives for using SRSs in any teaching scenario (Lai and Hong, op.cit.). Importance of feedback Apart from the benefits already mentioned, Kahoot! enables teachers to provide feedback by assessing students’ responses almost immediately. Feedback has been identified as key to successful learning in many learning theories (Hedgcock and Rouwenhorst, op.cit.). In the case of this SRS, the feedback is instant and there are also opportunities to provide more detailed and targeted feedback or, in response to students’ answers, to spend more time on a particular theory, model or concept. This means that teachers need to be flexible in their delivery of the material and adopt an ad hoc approach. Research has also revealed that students are more likely to change their study habits when early and frequent feedback demonstrates that their current study habits are inadequate (Love, Love, and Northcraft 2010). Unlike less expensive techniques, such as raising hands, these so-called ‘clickers’ can provide a degree of anonymity, real-time tallying and response-presentation that can improve the overall effectiveness of feedback (Hedgcock and Rouwenhorst, op.cit.). Clickers also force students to pick an answer themselves instead of potentially relying on others, as Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 in the case of having to raise their hands. This has implications for the building of self- confidence. According to Bandura (1986), success achieved on one’s own is the strongest source of ‘self-efficacy’ (Hedgcock and Rouwenhorst, op.cit.). Anecdotally, the results of the National Student Survey (NSS) have revealed that students lack clarity about when and how they receive informal feedback. Furthermore, Hedgcock and Rouwenhorst (op.cit., p.16) reported that “when clickers were used to facilitate feedback, students reported a better understanding of the materials, read more chapters before class, were more likely to recommend the course to others, and had higher exam scores than when clickers were used just for attendance”. This indicates the possibility of overall improvements in students’ learning, performance and attitudes. Learning Theories There are two main learning theories that can help to underpin the use of Kahoot! and other SRSs (Hedgcock and Rouwenhorst, op.cit., pp.16-17). Behaviourism and cognitive theories reinforce and contextualise the appropriateness of using such digital technologies to consolidate student learning. Behaviourism declares that learning results from a change in observable behaviour, where the stimuli cause the learner to respond in a certain way (cf. Bush, 2006; Weegar and Pacis, 2012). In this case, the student follows the instructions, takes the quiz, is likely to be motivated by scores (which are recorded and shown after each question is answered) and, as a result, makes gains in knowledge. Cognitive theories state that learning is a process of gaining or changing insights, expectations or thought patterns and developing capacity and skills to learn better (Hedgcock and Rouwenhorst, op.cit.). In order for learning to occur, there are nine elements that needs to be present according to Gagné, Briggs and Wager (1992). These are, in a consecutive order: (1) gaining attention; (2) informing learners of the objective; (3) stimulating recall of prior learning; (4) presenting the stimulus; (5) providing learning guidance; (6) eliciting performance; (7) providing feedback; (8) assessing performance; and (9) enhancing retention and transfer (Matthews et al. 2012, p.72). From these elements, providing adequate feedback – in other words ‘reinforcement’ – is the most significant one associated with student success (Martin, Klein and Sullivan, 2007). Thus the feedback loop or reinforcement is key to ensuring that students receive immediate feedback and are able to learn from it. Description of the case study and Implementation For the purpose of this small-scale, exploratory study, Kahoot! was introduced during the delivery of material for Research Methods tutorials to fifteen second-year hospitality management undergraduate students in term 2 for twelve weeks. The one-hour tutorial, which followed a one-hour lecture, took place at 2 p.m. on Wednesday afternoons. The main rationale for using this particular type of SRS was to test students’ understanding of the lecture material, increase the class engagement, create a better learning experience with this gamified activity, clarify any misunderstandings and bring some fun and excitement into the classroom environment. The author designed the quizzes to test students’ knowledge and understanding of what had been covered during the previous lectures and tutorials. Short, ten-question Kahoot! quizzes were carried out – in only some tutorials, so as to engender some spontaneity; the lecturer decided always to run the quizzes at the start of the tutorial, rather than during or after, as a method of finding out if students were ready to move on with the material. An example of questions and answers for a quiz on a qualitative research design can be found in Table 1. Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 Table 1. Example of questions and answers for a quiz on qualitative research design That the students in this study had not used this SRS before the author introduced it to them was another reason for its implementation. Clear instructions and assistance were provided to students on how to use their internet- enabled devices to access the correct web address. On the first occasion, students practised completing a quiz, before taking the actual quiz with recorded scores. Each time students answered questions incorrectly, the author would clarify this immediately and offer a clear explanation behind the correct answer. Winners of the three top places were always asked to reveal their identity and the class would congratulate them on their high scores with a round of applause. This further boosted class morale and peer-to-peer engagement. The lecturer would offer some time for peer discussion after the quiz to answer questions or clarify incorrect answers. Method of evaluation At the end of this twelve-week course, all fifteen hospitality management students were asked to complete an anonymous, six-question, Microsoft Forms online survey, containing both, closed and open-ended questions (Table 2). Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 Table 2. Survey Questions 1. Kahoot! quizzes made the tutorial more engaging. 2. Kahoot! quizzes have improved my understanding of the research methods material. 3. After completing a Kahoot! quiz I felt more confident about my knowledge of the subject. 4. I enjoyed Kahoot! quizzes during Research Methods tutorials because.... 5. I did not enjoy Kahoot! quizzes during Research Methods tutorials because.... 6. Please leave any other comments or feedback here regarding the use of Kahoot! quizzes during Research Methods tutorials. Questions 1-3 (whose aim was to find out students’ attitudes towards Kahoot!) were informed by the reviewed literature and, in particular, the research by Nielsen et al. (op.cit.) and Hedgcock and Rouwenhorst (op.cit.). The remaining three questions were designed to gain a more in-depth understanding of students’ perceptions of the use of Kahoot! during tutorials and, more specifically, the extent to which they enjoyed these quizzes. To the first three questions, students provided responses on a Likert scale, choosing between five options from ‘strongly disagree’ to ‘strongly agree’. Questions 4-6 were open-ended. Students were informed about the purpose of this short survey and by completing it anonymously they consented to taking part in this exploratory research. The achieved response rate of 33% was relatively low, which could have been caused by the time of the survey’s distribution (during the busy term 1 of students’ final year). Presentation and analysis of the data The following section presents and analyses the data gathered via an online survey. For the full set of responses to questions 1-6, please see Appendix 1. The majority of students appeared very enthusiastic about taking part in Kahoot! quizzes and particularly liked the leader board, which brought an element of competition as this comment indicates: “This method was one of the best ideas used during the research methods' tutorials, not just because it made us ‘compete’ by putting our knowledge, our understanding and learning skills into the actual class, but more important because it made us to be more engaged with this course” Moreover, the overall classroom atmosphere improved and students felt confident to ask for further clarification on certain concepts after providing incorrect answers and receiving immediate feedback. In their responses, students stated that the quizzes made the tutorials more engaging, with the overall score of 4.8 out of 5 (question 1). The direct quotes below help to illustrate this finding. “It's funny, engaging and good exercise for our knowledge” “I was more engaged as it had some fun in it than the usual traditional teaching structure” Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 “It was a fun and engaging way to learn a subject which was difficult to get my head around in the early stages” When students feel more engaged with the learning situation, they are more likely to work hard to make sense of the course materials and therefore more likely to perform better on assessments measuring learning (Mayer et al., 2009). Moreover, this technique can also motivate students to participate in class and take more responsibility for their learning. Students also agreed that the quizzes improved their understanding of research methods, with the overall score of 4.4 out 5 (question 2) as seen below in the following direct quotes: “It represents a fun and interactive method of learning and understanding the information that is provided” “It made me understand the subject better and made me more confident on my knowledge” This corresponds with a number of research findings which state that using such learning technologies helps students to get a better understanding of the material (Hedgcock and Rouwenhorst, op.cit.), including better conceptual understanding (Crouch and Mazur, 2001). According to Viberg and Grönlund (2017), such informal learning practices can also improve students’ confidence and competence. Having completed Kahoot! Quizzes, students felt rather more confident about their knowledge of the subject, with the overall score of 3.8 out 5 (question 3). This suggests not only an increase in their confidence about the material studied, but also their self-confidence, as illustrated in the quote below: “I enjoyed the use of Kahoot! because you get to realise that you’re not the only person amongst the class that does not understand a particular area therefore you’re not afraid to ask questions as other colleagues may have the same problem. It would be great to use Kahoot! in class again” It is therefore critical for the teacher to react to incorrect responses, so as to clarify any possible misunderstandings and to demonstrate the point concerned and explain that students’ current knowledge is inadequate. At the same time, those students who provide correct answers should be praised for their high scores. Limitations, conclusions and recommendations The aim of this case study was to demonstrate and share the author’s evaluation of using Kahoot!, a type of ‘student response system’, during Research Methods tutorials delivered to fifteen second-year hospitality management students. As many young people have a significant amount of technology experience even before they enter university (Lai and Hong, op.cit.), digital technologies can be utilised in a classroom setting to support various learning practices (Viberg and Grönlund, op.cit.). Thus, the motivation for using this learning technology was to reinforce the key concepts, often very abstract, consolidate students’ learning, provide them with real-life feedback and help them to grow in their knowledge of and confidence in engaging with the subject matter. This case study, based on a small-scale exploratory research sample, showcases the benefits of incorporating this SRS in the Research Methods tutorials. These benefits, as student feedback suggests, include better Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 conceptual understanding and increased motivation, resulting in the transition from surface to deep learning Furthermore, students enjoyed completing the Kahoot! quizzes, which created a lively learning environment where they felt confident and safe enough to admit if they needed further clarifications. There are some limitations to this case study. To avoid a low response rate in an online survey, the teacher should consider collecting during sessions the informal feedback from students regarding their experiences of using such an SRS as Kahoot!, as well as dedicating a final session for collecting anonymous online feedback. To increase the sample size and make the findings generalisable, feedback should be collected from every participating cohort of students. This case study has focused mainly on the benefits of using this type of SRS and paid less attention to possible challenges and to negative impacts on student learning. Future research should consider this angle. To ensure spontaneity, such technology should not be so overused that students feel demotivated to attend tutorials. Furthermore, practical issues need to be considered, such as cost, commitment to using SRSs and possible technological problems. It is also imperative for student learning that there is sufficient amount of time left after each quiz for questions and feedback. Being mindful of short delivery time, it is recommended that each quiz is no more than ten questions long and that sufficient time is given to answer each question. The ideal time to answer each question can be determined during ‘the practice run’; this will leave enough time for making use of incorrect answers to aid discussion and continue with other tutorial material. Teachers should feel fully prepared, confident and have a ‘plan B’ once they have identified clear goals and motivation for implementing any SRS. According to Nielsen et al. (op.cit.) a teacher’s commitment has been found as the most important factor for a successful implementation of SRS. Apart from deploying Kahoot! for informal learning practices, the use of this type of SRS is recommended as part of summative assessments and out-of-class exercises. Although existing research suggests that students are very familiar with digital technologies and confident about using them, teachers should not assume that all students have the same learning capabilities. Instructions and support should therefore be provided to all students before playing each Kahoot! Quiz, to ensure student needs are recognised and met. On the basis of the findings of this case study, the author also recommends that teachers in the HE sector should consider introducing Kahoot! to support their teaching on the assumption that classrooms are equipped with the AV system and that students have access to internet- enabled devices. In terms of student numbers, Kahoot! quizzes can be played in a lecture setting where more than fifteen students are present. The software offers a team mode, as well as the individual player mode, which means that students could work in small groups to complete the quiz. This particular SRS was used during the Research Methods tutorials, but there are opportunities for its implementation in other subject areas. If the teacher wants to test students’ knowledge, reinforce key concepts, increase students’ confidence in the subject, offer immediate feedback and clarification and introduce fun and excitement to the classroom, Kahoot! quizzes are applicable to any discipline. However, further research is needed to determine the extent of that. Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 Reference list Bandura, A. (1986) Social Foundations of Thought and Action: A Social Cognitive Theory, Englewood Cliffs: Prentice-Hall. ISBN 9780138156145 Boyle, J.T. and Nicol, D.J. (2003) ‘Using classroom communication systems to support interaction and discussion in large class settings.’ Association for Learning Technology Journal, 3(11), 43-57. Available at: https://journal.alt.ac.uk/index.php/rlt/article/view/1093/1343 (Accessed: 10 February 2018). Brenner, T. (2015) The use of Mobile Devices in the College Classroom. Available at: http://bokcenter.harvard.edu/blog/use-mobile-devices-college-classroom Harvard University: The Derek Bok Center for Teaching and Learning (Accessed: 16 October 2017). Brenton, S. (2009) ‘E-learning – an introduction.’ In: Fry, H., Ketteridge, S. and Marshall, S. (eds.), A handbook for teaching and learning in higher education. Oxford: Routledge, 85-98. Bush, G. (2006) ‘Learning about learning: from theories to trends.’, Teacher Librarian, 34(2), 14-19. Available at: https://search.proquest.com/openview/ad532d45ee7862958c5f528a81821611/1?pq- origsite=gscholar&cbl=38018 (Accessed: 10 March 2018). Caldwell, J.E. (2007) ‘Clickers in the large classroom: current research and best-practice tips.’ CBE - Life Sciences Education, 6, 9-20. Available at: https://doi.org/10.1187/cbe.06-12- 0205 (Accessed: 20 April 2018). Compton, M. and Allen, J. (2018) ‘Student Response Systems: a rationale for their use and a comparison of some cloud based tools.’, Compass: Journal of Learning and Teaching, 11(1), 1-19. Available at: https://journals.gre.ac.uk/index.php/compass/article/view/696/pdf (Accessed: 5 April 2018). Crouch, A.H. and Mazur, E. (2001) ‘Peer instruction: ten years of experience and results.’ American Journal of Physics, 69(9), 970-977. Available at: http://web.mit.edu/jbelcher/www/TEALref/Crouch_Mazur.pdf (Accessed: 5 April 2018). Draper, S.W. and Brown, M.I. (2004) ‘Increasing interactivity in lectures using an electronic voting system.’ Journal of Computer Assisted Learning, 20, 81-94. Available at: http://www.psy.gla.ac.uk/~steve/evs/papers/draperbrown.pdf (Accessed: 15 March 2018). Gagne, R., Briggs, L. and Wager, W. (1992) Principles of Instructional Design (4th Edition). Fort Worth, TX: HBJ College Publishers. ISBN 0-03-034757-2 Hedgcock, W.H. and Rouwenhorst, R.M. (2014) ‘Clicking their way to success: using student response systems as a tool for feedback.’ Journal for Advancement of Marketing Education, 22(2), 16-25. Available at: http://www.mmaglobal.org/publications/jame/current- past-issues/“http:/www.mmaglobal.org/publications/JAME/JAME-Issues/JAME-2014-Vol22- http://bokcenter.harvard.edu/blog/use-mobile-devices-college-classroom https://doi.org/10.1187/cbe.06-12-0205 https://doi.org/10.1187/cbe.06-12-0205 http://www.mmaglobal.org/publications/jame/current-past-issues/%E2%80%9Chttp:/www.mmaglobal.org/publications/JAME/JAME-Issues/JAME-2014-Vol22-Issue2/JAME-2014-Vol22-Issue2-Hedgcock-Rouwenhorst-pp16-25.pdf http://www.mmaglobal.org/publications/jame/current-past-issues/%E2%80%9Chttp:/www.mmaglobal.org/publications/JAME/JAME-Issues/JAME-2014-Vol22-Issue2/JAME-2014-Vol22-Issue2-Hedgcock-Rouwenhorst-pp16-25.pdf Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 Issue2/JAME-2014-Vol22-Issue2-Hedgcock-Rouwenhorst-pp16-25.pdf (Accessed: 19 October 2018). Kivunja, C. (2014) ‘Theoretical perspectives of how digital natives learn.’ International Journal of Higher Education, 3(1), 94-109. Available at: https://doi.org/10.5430/ijhe.v3n1p94 (Accessed: 15 June 2018). Lai, K.-W. and Hong, K.-S. (2015) ‘Technology use and learning characteristics of students in higher education: Do generational differences exist.’ British Journal of Educational Technology, 46(4), 725-738. Available at: https://doi.org/10.1111/bjet.12161 (Accessed: 15 June 2018). Love, E.G., Love, D.W. and Northcraft, G.B. (2010) ‘Is the end in sight? Student regulation of in-class and extra-credit effort in response to performance feedback.’ Academy of Management Learning & Education, 9(1), 81-97. Available at: https://doi.org/10.5465/amle.9.1.zqr81 (Accessed: 15 June 2018). Martin, F., Klein, J. and Sullivan, H. (2007) ‘The impact of instructional elements in computer-based instruction.’ British Journal of Educational Technology, 38(4), 623 - 636. Available at: https://doi.org/10.1111/j.1467-8535.2006.00670.x (Accessed: 15 June 2018). Masikunas, G., Panayiotidis, A. and Burke, L. (2007) ‘The use of electronic voting systems in lectures within business and marketing: a case study of their impact on student learning.’ Research in Learning Technology, 15(1), 3-20. Available at: https://doi.org/10.1080/09687760600837090 (Accessed: 2 July 2018). Maton, K. and Kervin, L. (2008) ‘The digital natives debate: A critical review of the evidence.’ British Journal of Educational Technology, 39(5), 775-786. Available at: https://doi.org/10.1111/j.1467-8535.2007.00793.x (Accessed: 20 May 2018). Mayer, R. (2005) The Cambridge Handbook of Multimedia Learning, Cambridge: Cambridge University Press. ISBN 9780521838733. Nielsen, K.L., Hansen, G. and Stav, J.B. (2013) ‘Teaching with student response systems (SRS): teacher-centric aspects that can negatively affect students’ experience of using SRS.’ Research in Learning Technology, 21, 1-13. Available at: http://dx.doi.org/10.3402/rlt.v21i0.18989 (Accessed: 20 May 2018). Prensky, M. (2001) ‘Digital natives, digital immigrants.’.On the Horizon, 9(5), 1–2. Available at: https://doi.org/10.1108/10748120110424816 (Accessed: 20 May 2018). Rao, S.P. and DiCarlo, S.E. (2000) ‘Peer instruction improves performance on quizzes.’ Advances in Physiology Education, 24, 51-55. Available at: https://doi.org/10.1152/advances.2000.24.1.51 (Accessed: 20 May 2018). Rice, R.E. and Bunz, U. (2006) ‘Evaluating a wireless course feedback system: the role of demographics, expertise, fluency, competence, and usage.’ Studies in Media and http://www.mmaglobal.org/publications/jame/current-past-issues/%E2%80%9Chttp:/www.mmaglobal.org/publications/JAME/JAME-Issues/JAME-2014-Vol22-Issue2/JAME-2014-Vol22-Issue2-Hedgcock-Rouwenhorst-pp16-25.pdf https://doi.org/10.5430/ijhe.v3n1p94 https://doi.org/10.1111/bjet.12161 https://doi.org/10.5465/amle.9.1.zqr81 https://doi.org/10.1111/j.1467-8535.2006.00670.x https://doi.org/10.1080/09687760600837090 https://doi.org/10.1111/j.1467-8535.2007.00793.x https://doi.org/10.1108/10748120110424816 https://doi.org/10.1152/advances.2000.24.1.51 Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 Information Literacy Education, 6(3), 1-32. Available at: http://www.edutekmidwest.com/wp- content/uploads/2011/11/Course_feedback_system.pdf (Accessed: 5 June 2018). Šorgo, A., Bartol, T., Dolničar, D. and Podgornik, B. (2017) ‘Attributes of digital natives as predictors of information literacy in higher education.’ British Journal of Educational Technology, 48(3), 749-767. Available at: https://doi.org/10.1111/bjet.12451 (Accessed: 5 June 2018). Viberg, O. and Grönlund, Å. (2017) ‘Understanding students’ learning practices: challenges for design and integration of mobile technology into distance education.’ Learning, Media and Technology, 42(3), 357-377. Available at: https://doi.org/10.1080/17439884.2016.1088869 (Accessed: 20 May 2018). Weegar, M.A. and Pacis, D. (2012) ‘A Comparison of Two Theories of Learning-Behaviorism and Constructivism as applied to Face-to-Face and Online Learning.’ In: Proceedings E- Leader Conference, Manila. Available at: https://pdfs.semanticscholar.org/2cb0/b3019f04cfbf790d0ed8fa39f603bcfa4f7e.pdf (Accessed: 20 May 2018). https://doi.org/10.1111/bjet.12451 https://doi.org/10.1080/17439884.2016.1088869 Case Studies Compass: Journal of Learning and Teaching, Vol 11, No 2, 2018 Appendix 1. Full set of responses to an online survey Survey Questions Responses 1. Kahoot quizzes made the tutorial more engaging. 4.8 out 5 average rating 2. Kahoot quizzes have improved my understanding of the research methods material. 4.4 out 5 average rating 3. After completing a Kahoot quiz I felt more confident about my knowledge of the subject. 3.8 out of 5 average rating 4. I enjoyed Kahoot quizzes during Research Methods tutorials because.... “It's funny, engaging and good exercise for our knowledge.” “It represents a fun and interactive method of learning and understanding the information that is provided.” “I was more engaged as it had some fun in it than the usual traditional teaching structure.” “It was a fun and engaging way to learn a subject which was difficult to get my head around in the early stages.” “It made me understand the subject better and made me more confident on my knowledge.” 5. I did not enjoy Kahoot quizzes during Research Methods tutorials because.... “I enjoyed them.” “Nothing. This is not true. As I mentioned I did enjoy the Kahoot quizzes. Thank you!!” “N/A” “I remember correctly, it didn't give you a lot of time to answer the questions.” “Not applicable. I did enjoy it.” 6. Please leave any other comments or feedback here regarding the use of Kahoot quizzes during Research Methods tutorials. “This method was one of the best ideas used during the research methods' tutorials, not just because it made us "compete" by putting our knowledge, our understanding and learning skills into the actual class, but more important because it made us to be more engaged with this course.” “I enjoyed the use of Kahoot because you get to realise that you’re not the only person amongst the class that does not understand a particular area therefore you’re not afraid to ask questions as other colleagues may have the same problem. It would be great to use Kahoot in class again.”