TF_Template_Word_Windows_2013 Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 1 An investigation into digital tools for lecture engagement: a feasibility study Michael Detyna1, Eleanor J. Dommett2 1(Corresponding author), Centre for Technology Enhanced Learning, King’s College London, UK. 2Dept of Psychology, Addison House Guy’s Campus, King’s College London, UK Abstract Evidence suggests that lectures are of most value in higher education when they are interactive and support active learning (Freeman et al., 2014). Using novel approaches within lectures can help go beyond the traditional university experience. Educational technologies offer several options for supporting this, including audience response systems, backchannel communication, mirroring and use of video. However, given the range available and the cost of implementation, it is important to ensure that the right technologies are adopted. The aims of this study were 1) to investigate the feasibility of small group sessions to evaluate the use of specific technologies for lectures and 2) to understand better the potential uses of different technologies for lectures. Staff and students participated in a novel approach with hands-on interactive demonstration sessions before taking part in a focus group to give their views on a variety of technologies. The current study found that these small-scale interactive demonstrations were an effective way to evaluate technologies and that several of the technologies presented could be used either 1) to enhance current lecture practice or 2) to support new practice, provided they did not overwhelm or distract students. However, they must also be simple for staff and students to use. Keywords: Learning technology, lectures, pedagogy, audience response systems, backchannel communication, mirroring, video. Introduction The COVID-19 outbreak has affected the whole world (Saxena, 2020) and led to a move away from face-to-face teaching and towards online teaching (Sun et al., 2020). However, once the pandemic is brought under control, lectures will continue to be – as they were before it – the dominant mode of instruction for university teaching. The research detailed here was conducted prior to the pandemic and aimed primarily at investigating digital tools available for those physically in lectures, although the work also has implications for blended learning and other approaches. Further, the three knowledge domains of pedagogy, content and technology should not be seen in isolation, but as part of a framework described eloquently by Mishra et al. (2009). Lectures provide the most economical approach to teaching large classes and are ingrained in the culture of academia. Research shows that students value lectures highly, reporting that they feel involved in the learning process and can engage in independent thinking and problem solving during lectures (Covill, 2011). Problem-based learning is a strategy that can be used effectively, although it is not without its challenges as Huijser, H. et al. (2016) Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 2 discuss. Furthermore, studies have found that lectures can result in effective learning in interactive classrooms (Van Dijk, Van Der Berg and Van Keulen, 2001), provide an appropriate forum for modelling how experts approach tasks (Feldon, 2010), support time management and enable the development of affective learning (Titsworth, 2001). However, there is also research showing lectures to be unhelpful, resulting in higher failure rates, reduced engagement and increased boredom when compared with other teaching methods (Kelly et al., 2005; Mann and Robinson, 2009; Schmidt et al., 2015). In recent years, such research has, in part, resulted in the stigmatisation of lectures (DiPiro, 2009; Gross-Loh, 2016). Despite this stigmatisation, increasing student numbers and limited classroom space in many universities mean it is likely that lectures are here to stay. It is therefore important to optimise the lecture by maximising student engagement and supporting active learning wherever possible (Freeman et al., 2014). An interactive lecture which provides opportunities for active learning could be a very powerful learning tool, allowing students to engage directly with material and build new knowledge into their existing frameworks (Bain, 2011; Mallin, 2017; Stacy, 2009). One way to support interactivity is by using educational technologies, the most prominent of which, used in lectures, is undoubtedly ‘student response systems’ (SRSs). Research into early SRSs has revealed positive attitudes towards them (Gaddis et al., 2006; Lin, Liu and Chu, 2011), beliefs that they support engagement and active learning (Kaleta and Joosten, 2007) and, consequently, improved performance (Hall et al., 2005; King and Joshi, 2006; Lyubartseva, 2013). There is less research into the web-based SRSs such as ‘Poll Everywhere’, which allows voting via text or online, but early evidence suggests similarly positive attitudes (Shon and Smith, 2011) and increased engagement (Gehlen-Baum et al., 2014; Kappers and Cutler, 2015). More gamified SRSs have also been found to have a positive effect on student engagement (Wang, 2015) and classroom dynamics (Licorish et al. 2018) in specific circumstances. Compton and Allen (2018) have provided a comprehensive review of different technlogies for SRSs. Current SRSs make use of the students’ personal mobile devices (smartphones, laptops) to engage with lecture-related activities, something which is thought to offset the potential distraction that they can create in a lecture (Fried, 2008; Kirschner and Karpinski, 2010). However, SRSs are not the only technology that can utilise these devices. Amongst many other uses, mobile devices can also be used for wireless mirroring and recording of a broadcast computer screen, so that students can view the lecturer’s computer screen on their own devices. Additionally, interactivity can be achieved with backchannel communication, which can use technologies and skills that, as students report, they frequently use (Fiester and Green, 2016). Tools available for backchannel communication include Padlet, a free online technology – which acts like a bulletin board and can be integrated into the virtual learning environment (VLE) – and Skype, both of which show potential for application in other types of learning environment but have received little attention in the lecture setting to date (Dunbar, 2017; Gill et al., 2014). Of the various technologies that students report that they use to support their learning, the most frequent is YouTube (Gill et al., 2014). Research also suggests that students value the inclusion of video clips in lectures (Eick and King Jr, 2012; Mitra et al., 2010). Whilst YouTube does contain a range of resources, there are other services which may be of use, Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 3 including an on-demand television broadcast video service, such as ‘Box of Broadcast’, which can show clips in lectures. Though it is quite common for videos to be deployed within lectures, there has been little formal evaluation of their impact. Such videos are clearly popular, but the technology has advanced significantly and there is now the possibility of using immersive video, augmented or virtual reality in lectures, (Stojšić et al. 2018, Detyna et al. 2019). From this brief review of the literature concerning educational technologies in lectures, it is apparent that they offer great potential for optimising interactivity in that context. However, the potentially high cost – both of implementation of these technologies and the training required to use them – makes it important to establish whether key stakeholders in the lecture consider them to be of any value before their application is rolled out more widely. The aims of this study were 1) to investigate the feasibility of small-scale demonstration sessions to evaluate the use of specific technologies for lectures and 2) to understand better what might be possible in deploying different technologies within lectures. For the latter, we were specifically interested in 3) how they could support teaching and learning and 4) any pros and cons of each technology. Materials and methods Ethical approval Ethical approval was obtained from the Institutional Ethics Committee (MR/16/17-744). All participants were then given printed study information and they provided written consent to participate. Participants Participants (N=33), of whom seventy per cent were staff, were recruited via advertisements on posters and also the institutional VLE, where they could voluntarily sign up to attend a session – staff and student sessions being held separately. Here we report data only on technologies reviewed by both staff and students to allow comparison between them. Aware that the larger the group size we had, the less each individual participant would be involved in discussion, we aimed for smaller groups of between three and six to allow richer feedback; again, staff and student sessions were separate. Research design and procedure As it was anticipated that most of the participants would not be familiar with the technologies being examined, they were provided with a session intended to ensure that – just as students would – they would see and interact with available tools in an appropriate learning environment; the chosen context was an introductory lecture on the science of the stars, where learning about the physics and chemistry of stars and their elements could add an educational background to the session. The topic of this content (known as stellar nucleosynthesis) helps answer the questions ‘How did the stars form?’, and ‘Where do the elements come from?’ and would be new to most participants. Following the session, collection of qualitative data ensued, by means of small focus groups that allowed everyone the chance to speak. Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 4 Participants attended a two-hour session, divided into three parts: 1. Lecture demonstration, using five different technologies to provide direct experience of the technology in context (twenty minutes). 2. Interactive opportunity, during which participants were invited to interact with the individual technologies, including attempting to set them up (sixty minutes). 3. Feedback period, in which the participants evaluated each tool with answers to the question ‘This tool could be useful for teaching’ on a Likert scale ranging from ‘strongly disagree’ (1) ‘to strongly agree’ (5). They then took part in a focus group discussion, which revolved around the potential value of the tools, so as to understand 1) the learning goals that different technologies could support; 2) how they could be used in a teaching environment and 3) the perceived pros and cons of each technology. Discussion of each area was prompted by a question – e.g. (for area 1) ‘What learning goals could this tool help you achieve?’. Follow-up questions for each area concerned staff and student perceptions or asked participants to review more deeply their rationale – e.g. ‘Why do you think this?’ or ‘How might this work in your discipline?’. The focus group discussion was audio-recorded for later analysis. Different technologies were demonstrated to both staff and students over a series of sessions, so that no one participant was exposed to all. These technologies were selected to include 1) SRS (Poll Everywhere and Kahoot!) 2) backchannel communication technologies (Padlet, Skype) 3) mirroring technologies (Mirroring 360) and 4) video technologies, including immersive video (Box of Broadcast, 360-degree video). A summary of the functionality of these technologies is provided in table 1. Table 1: A description of the technologies evaluated by both staff and students Technology System Description Audience response system Poll Everywhere Enables staff to engage with a class via real-time online feedback. Students respond in real time to questions via mobile device. Kahoot! A game-based learning platform where students are able to answer – in real time – a quiz, poll or survey. Backchannel communication Padlet An application designed to create online bulletin boards that allow students to share, via mobile device, a variety of content, including questions, discussion comments and multimedia. Skype Video chat platform. The case for proposed use here is video chat to enhance lectures, through dialogue, in a visual and interactive manner. Mirroring Mirroring 360 Software that allows wireless mirroring and recording of a broadcast computer screen so that participants may view the lecturer’s computer screen on their own devices. Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 5 Video 360-degree video 360 videos take a series of video images from all angles. This creates an immersive video experience which can be seen from all angles. Box of Broadcasts A web-based, on-demand television broadcast video service which can show clips at desktop or in lectures. Some other tools, considered by staff only, include augmented and virtual reality; these will be considered in a separate paper. Data analysis Data from the Likert scale rating on usefulness was collated by technology and checked for normality using the Kolmogorov–Smirnov test, as well as a measure of skewness and kurtosis. Where a normal distribution was found, we compared individual technologies’ ratings between staff and students, using independent sample t-tests. For technologies where ratings were not normally distributed, comparisons were made using the Mann Witney U non-parametric test. In both cases, mean and standard deviation data are displayed to allow comparison across all technologies. The recordings of the focus groups were transcribed and analysed using a thematic analysis (Braun and Clarke, 2006) with a six-stage process (Clarke and Braun, 2013): familiarisation, coding, theme extraction, review, naming and narrative analysis. Quotes are provided as validity of evidence (Mays and Pope, 1995). Punctuation was added to unambiguous quotes, spelling mistakes were corrected and, where necessary, words were added in square brackets for clarification. Multiple quotes from one person were treated as a single comment to avoid over-representation of an individual. Initial coding was completed by one researcher and then reviewed by the second. Following the thematic analysis to understand staff and student perceptions of the tools and what considerations are important in selecting tools, transcripts were also reviewed to identify specific examples of use cases. Results Usefulness ratings Table 2: Combined staff and student ratings for the usefulness of the different technologies on a scale of 1-5, where 5 indicates strong agreement that the technology would be useful in teaching and 1 indicates strong disagreement. Technology Rating (Mean ± SEM) Poll Everywhere 4.71 ± 0.18 Skype 4.22 ± 0.15 Box of Broadcast 4.00 ± 0.00 Padlet 3.91 ± 0.29 Mirroring 360 3.78 ± 0.22 Kahoot! 3.5 ± 0.261 360-degree video 3.22 ± 0.43 Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 6 Figure 1: Staff and student responses to the question ‘This technology could be useful for teaching’ on a Likert scale. Thematic analysis Three different themes emerged from the dataset: Theme 1: Pedagogic benefit, which could further be divided into a) optimising existing practice and b) supporting new approaches; Theme 2: Ease of use for both the staff and students; Theme 3: Avoiding overload and distraction. Theme 1: Pedagogic benefit Pedagogic benefit was identified as key by both staff and students. One way in which this benefit could be realised was in optimising current practice. Both staff and students identified several ways by which this could arise. Firstly, it was suggested that the technology could increase participation: ‘There is more participation in the class – everyone gets a chance to participate, it’s not just one person at a time.” [Student, Padlet] “It would be good for students who are nervous about talking up. I think it would be good for non-fact-based learning.” [Staff, Padlet] “I see it as enhancing engagement rather than delivering learning goals, but certainly could be used to reinforce key learning goals especially by high quality productions.” [Staff, Box of Broadcasts] Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 7 Secondly, it was suggested that some of the different technologies could allow students to visualise key concepts: “It gives students the ability to access related material to the lecture – could ask students to watch things after a lecture to learn more (e.g. a performance, video etc.)” [Staff, Box of Broadcasts] “I would use it to visualise a difficult concept – used sparingly students would better recall the concept by taking a link to a high quality [broadcast] production” [Staff, Box of Broadcasts] ‘Sometimes it can be difficult to talk about something in a simple and engaging way, and having it with short clips when you take a clip from a documentary with a high quality production team with text that has been well thought through would be useful. So long as it’s not overused.” [Staff, Box of Broadcasts] “It can help visualize things more, and can help see another part of the world, and back up your points.” [Student, Box of Broadcasts] “It could be useful having students click and move around and experience an area.” [Staff, 360-degree video] “Content could be uploaded to [our VLE] and [students] asked to explore or answer relevant questions.” [Staff, 360-degree video] Thirdly, it was thought that a benefit to current practice could arise from using the tools to identify any misconceptions by testing the general level of understanding during the lecture: “One of the most helpful parts is for understanding common misconceptions that the students have regarding key topics. The way I use it in a lecture is to focus on the closest correct answer to the question and explain why this is incorrect.” [Staff, Poll Everywhere] “I would use it for a recalibration of the room, to see ‘what did we all learn today?’, and it allows the instructor to check what we all learned today without guided learning. To see what some of the issues are if there are issues, if there are any misguided thoughts" [Staff, Padlet] “[It would be] useful if there was a diagram and there could be a check if we were paying attention” [student, Poll Everywhere] Finally, it was raised that the technologies could overcome some physical barriers to learning. “I think it would be good for visibility in the lecture room, if there was a pillar in the way, or if people are unable to see it clearly” [Staff, Mirroring360] Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 8 As well as its providing pedagogic benefit by optimising current practice, several participants suggested ways in which the technology could support a new practice. One example of this was through working and learning as a peer group using Padlet and Mirroring 360: “I like that you could add and build on what other people have said, so it’s like peer to peer feedback, and it’s all instantaneous” [Student, Padlet] ‘Several groups could have separate discussions, and could add and expand on what people have said” [Student, Padlet] ‘Seeing the variety of responses allows opportunities for peer learning and self-assessment because students have immediate access to a wide spectrum of responses from classmates rather than a few responses from the vocal ones.” [Staff, Padlet] “I like it for this reason - it could be used for situations where you get students to present.” [Staff, Mirroring360] “I thought it would be good for group work…ok we’ve sat there and written down, rather than on paper onto a computer and then it can go up onto the main screen at the end and share it with the rest of the room…it depends, if you are in a lecture theatre with 400 students you do not end up with lots of discussion time and therefore that opportunity [for group work where you would want to share your thoughts]” [Student, Mirroring360] A second example that emerged from the data was the use of backchannel communication technologies to bring in expertise, for example, that of guest speakers. Staff comment as follows: “I definitely see a use case using it for bringing in an expert” [Staff, Skype] “I like the idea of guest speakers. Because we have industry speakers come in to talk to our students.” [Staff, Skype] “[An expert] can skype in from their offices and they are calling from a tablet or a phone they can give you a tour of their office, and show you what say Google is like from the inside”. [Staff, Skype] “It’s the idea of having an external speaker that doesn’t have the time to come in house, but could give a short presentation with you and your students, would be useful and value added.” [Student, Skype] “Asking an expert” or “real life” on site reporting/interview could bring a topic to life.” [Student, Skype] Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 9 Theme 2: Ease of use Both staff and students commented on the ease of use of technologies, with several perceived positively because they were simple to use: ‘simple and effective” [Student, Poll Everywhere] “It was very quick as soon as you logged in to the website it was just on, which I think was very good.” [Staff, Mirroring360] By contrast, there were concerns where a technology was more complex to set up: ‘There are a lot of steps to getting ready compared to say Poll Everywhere, so I’d be reluctant to do this if students are already used to Poll Everywhere.” [Staff, Padlet] ‘The only issue is the amount of preparation in advance. But I really like the idea of people putting ideas and putting more information up there.” [Staff, Padlet] “I think I’d [worry I’d] be standing there for half an hour pressing buttons. I don’t think I could use it quickly.” [Staff, general] ‘I struggle to see the practicalities of using it., I can’t see how it would slot in easily’ [Student, general] “Easy to use, when in a teaching environment you need to just get on.” [Staff, general] Related to this, staff also felt that having the same technology available to them in every teaching space made things easier for them: “Having everything set up in a way that you’re used to makes it easier to start the lecture” [Staff] Theme 3: Avoiding overload and distraction It became apparent that both staff and students were concerned about potentially overloading students or increasing distraction with technology: “Not sure what it offers over Poll Everywhere and worry about student overload” [Staff, Kahoot] “As long as it’s short and to the point it then it makes sense to use it.” [Student, Box of Broadcasts] “I don’t think it’s actually it is that useful. I struggle to see the practicalities of using it. Stuff like this, I can’t see how it would slot in easily […] and it could seem disjointed” [Student, 360-degree video] Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 10 “I don’t see that it adds a lot and it tends to break focus…the quality [of learning] can then drop because you are taking in so much information.” [Student, 360-degree video] Although, to counter this, the use of mobile devices for learning was seen as a way to reduce the distraction they might normally cause: “I felt that it was a good idea to embrace the fact that many students have smartphones and tend to look at them relatively often. I thought that using this system would also be a way where students could use their phones constructively" and take part in the lecture.” [Staff, Poll Everywhere] Sample use cases Following on from discussions with staff and students, several possible use cases were extracted from the transcripts (table 3). Note that this was only possible for technologies where comments were specific to the tool in question. In each case, an example use is supported by a quote from either staff or students. Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 11 Table 3: Example Use Cases for several technologies based on common suggestions from staff or students. Technology Use Cases Poll Every- where & Padlet Collaboration: Both tools allow students to share ideas e.g. through a word cloud in Poll Everywhere or through sharing more extensive text and images in Padlet. This can be an open share or directed by a resource or idea put up by staff: “I really like the idea of people putting ideas, and putting more in- formation up there.” [Staff] Checking understanding and gaining feedback: Both tools can be used to ask students questions and elicit their questions in a non-threatening way: ‘some [students] like the ability to give opinions without necessarily having to stick their heads above the parapet.” [Staff] “I would use it for a recalibration of the room, to see ‘what did we all learn to- day?’” [Staff] Skype Increasing expertise: This tool can be used for guest lecturers but also for smaller segments such as panel discussions after a lecture: “It’s the idea of having an external speaker that doesn’t have the time to come in house, but could give a short presentation with you and your students, would be useful and value added.” [Student] Virtual Field Trips: Skype could be used to have a tour of a space by an expert e.g. a researcher do- ing a lab tour: “[An expert] can skype in from their offices and they are calling from a tablet or a phone they can give you a tour of their office, and show you what say [a company such as] Google is like from the inside” [Staff] Collaboration & communication: Small groups of students can work together on projects using skype: “I know some people at another university that teach collaboratively, and students can be on that module, and they teach that via skype. Students have projects which are created collaboratively via Skype”. [Staff] Box of Broadcasts Flipped learning: Staff can select appropriate material for students to watch ahead of the face-to-face learning experience, during which more active discursive learning can then occur: ‘students could […] be asked to see a particu- lar Shakespeare performance and consider specific aspects of that performance whether it was costume or the way a scene was per- formed. Then in class we can review small chunks as a group after we have ruminated on it, and that can be really useful.” [Student] Supporting visualisation: Staff identify short video clip for use in lecture. ‘sometimes it can be difficult to talk about some- thing in a simple and engaging way, and having ... short clips when you take a clip from a documen- tary with a high-quality production team with text that have been well thought through would be useful.” [Staff] Students find own material: Staff choose appropriate topic and learning goals. Stu- dents search tool for relevant material. Stu- dents select relevant clips. Students share clips via institutional virtual learning envi- ronment (VLE) or in small group teaching. “It gives students the ability to access re- lated material to the lecture – could ask stu- dents to watch and find out things after a lecture to learn more (e.g. a performance, video etc.)” [Staff] Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 12 Discussion There were two distinct aims to this study. Firstly, we wanted to test the feasibility of a single session, in which staff and students are given demonstrations of technology and the opportunity to interact with it, as a means of gaining insight into staff and student views about new technologies. Secondly, we wanted to understand better the potential uses of different technologies for lectures. As stated earlier, the three knowledge domains of pedagogy, content and technology should not be seen in isolation, but as part of a framework described eloquently by Mishra et al. (2009). This research attempted to consider digital technological tools in a specific pedagogical setting, with a similar level of content knowledge (relating to stars and stellar nucleosynthesis) that would provide a roughly equal benchmark for participants and draw together the three knowledge domains. We’ll begin by discussing the second aim, which will then contribute to our discussion of the first. The feedback on the different technologies was to some extent consistent with previous research. For example, Poll Everywhere was commented on generally positively by both staff and students (Gaddis et al., 2006; Lin et al., 2011; Pollock, 2005) and gained the highest ratings overall. Staff identified that this technology could be a way of putting mobile phones to good use rather than having them serve as a distraction (Fried, 2008; Kirschner and Karpinski, 2010). The second SRS, Kahoot!, was less well-received, seemingly suffering from comparison to Poll Everywhere because it was more complex to set up. Backchannel communication technologies were also well-received, with several different suggestions being made for their use. Interestingly, Padlet seemed to be grouped more closely with Poll Everywhere in terms of suggested uses and, although it was generally seen as positive, in line with previous literature (Dunbar, 2017), staff did raise concerns about the complex set- up required. This was also the only technology for which staff and student usefulness ratings significantly differed, with students rating it more positively. Based on the focus group remarks, it seems likely that this was because they were not concerned with setting it up, but only responding to it. Skype was seen as offering a way to engage with individuals outside the university, e.g. guest lecturers. Previous studies from a range of disciplines indicates the value of guest lecturers (Rowe, 2004; van Hoek, Godsell and Harrison, 2011). The current study suggests that, where timing is appropriate, Skype can offer an appropriate means for guest lecturers to deliver material. It should be noted that, while the term ‘backchannel communication’ is used in the literature to describe Skype, the feedback and use cases derived from the current study suggest that it would not primarily be used for this type of communication. Interestingly, staff also raised the possibility of the expert joining the session via Skype and giving a tour of her/his own environment. This would be perfectly possible with 360-degree video, but this tool was less positively received. One possible explanation for this is the simplicity of the technology. Unlike Skype, Padlet divided staff and students slightly, with the latter viewing it more positively, which echoes the finding of Betts and Garnham (2018) – that it can help engagement. The mirroring technology evaluated in the current study, Mirroring-360, has not previously been the focus of research in higher education and the current study shows that both staff and students could see a value in it. Both groups reported that it could be helpful for when students needed to feed back or engage with the whole room in some way. It was also Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 13 deemed relatively simple to use. Given this feedback, it would be pertinent to conduct more in-depth research into the potential uses in lectures of this technology, but also in small group work, which was identified as a possible use. The final two technologies evaluated were video technologies – and the two received quite different reviews. Box of Broadcasts was generally well received – perhaps not surprising given that the technology is centred on videos which are already known to be popular in teaching and learning, including lectures (Eick and King Jr, 2012; Gill et al., 2014; Mitra et al., 2010). Interestingly, both staff and students commented on the length of the video clip used and this is in line with previous research, with recognition that clips should be cut to show only appropriate material (Mitra et al., 2010). Despite the positive reviews of Box of Broadcasts, feedback was less positive about the 360-degree video, in terms of ratings and qualitative remarks, with concern expressed that it could be overwhelming. As indicated in the discussion above, staff and students recognised the significant pedagogic benefits that some technologies could offer and comments were balanced, suggesting that there is not a constant drive for new technology irrespective of its value. Critically, there was also a significant emphasis on the need to keep the technology easy to use. This need for simplicity suggests that one key focus in rolling out the use of any new technologies will be to ensure that adequate training, where appropriate, is made available for staff and students and that the simplest system possible is put in place. It is also apparent that any technologies must offer a clear pedagogic benefit. This can come through both optimising existing approaches in lectures and offering new opportunities, such as peer/group work. Overall, for most of the technologies examined, staff and students could see a pedagogic benefit to their use. However, it was also apparent that the tools needed to be straightforward to use. Furthermore, while students already possess a degree of digital literacy, as Fitzgerald et al. (2015) comment, it is important to build in mechanisms to increase their digital literacy to equip them for them for the future. We return now to the feasibility of this approach to eliciting stakeholder views. The single sessions ran effectively and participants reported finding them useful. As demonstrated, a substantial amount of data was obtained from the feedback part of the session, indicating that this approach could be helpful in gaining insight about staff and student views of technology. With the technological landscape constantly shifting, and what is appropriate in one year being less appropriate in another, it is necessary to engage in a regular dialogue about different technologies with those who will use them. This study intended to start with a wide range of prior experience, from novices to those more confident, and then ensure a more standard benchmark by providing a recent experience of tools/approach through direct interaction in a relevant session. In the two-hour session, sixty minutes were allocated for direct experience, which gave participants, on average, fifteen to twenty minutes’ direct experience of each tool, although in practice this varied as they spent time mainly on those they felt were of most value. While this experience per tool may not seem like a large amount of time, it should be seen in the broader context of the two hours they were spending looking at, discussing and considering all the tools. Further reflection in subsequent studies could establish whether this time should be increased, although the general consensus from participants was that they had sufficient time to experience the technologies, reflect on them and offer considered opinions about them. Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 14 The approach taken in the present study is a cost-effective way to gain useful insights on a relatively regular basis before investing significant resources into a particular technology. Although the approach described worked well and yielded interesting data, some limitations of the study must be noted. Firstly, the sessions were open to any staff and student at the university and, as is the case with open sessions, those attending were self-selecting and therefore, in this case, may have been particularly keen to learn about new technologies, with consequent slight bias to the results. However, the fact that both negative and positive comments were made suggests that, even with a self-selecting group, this approach can yield valuable information. Secondly, the sample size was small. However, recent guidelines for thematic analysis (Braun and Clarke, 2006; Fugard and Potts, 2015) have suggested that studies using participant-generated text should include ten-fifty participants, indicating that this sample size is sufficient; furthermore, the total number of words generated from the transcribed text was well over 104. Conclusion The present study has demonstrated a novel approach to showing that small group sessions allowing staff and students to receive a teaching demonstration together with the opportunity to explore and feed back on specific technologies can yield useful insights into the value of that technology for teaching. Data indicate that simple SRS and basic video tools such as Box of Broadcasts are well received. Backchannel communication technologies are also well-received (despite their suggested use not actually being for this kind of communication). Irrespective of the individual technologies, the evidence presented suggests that any technology implemented should have a clear pedagogic benefit, for example through increased engagement, the ability to test understanding or inclusion of peer interaction and guest lectures. For the future, a productive area of research could be evaluation of technologies which may be better suited for use in non-lecture teaching where the requirements may be slightly different. At present however, it is possible to conclude that specific technologies, when simple to use, are of benefit to large-scale teaching. This present study demonstrates that there are specific digital tools, particularly those most straightforward to use, that can increase engagement and are seen by both staff and students to have the potential to enhance learning. Acknowledgments We’d like to acknowledge James Toner, Phil Blake, Vaishnavi Gogu, Andre Crawford and Jerome Di Pietro and Dave Busson for support on this project. Declaration of interest statement No potential conflict of interest was reported by the authors. Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 15 Reference list Bain, K. (2011) What the best college teachers do. Cambridge, Mass: Harvard University Press. ISBN. 9780674013254 Betts, T. and Garnham, W. (2018) ‘The Padlet Project: Transforming student engagement in Foundation Year seminars.’ Compass: Journal of Learning and Teaching, 11(2). Available at: https://journals.gre.ac.uk/index.php/compass/article/view/714 (Accessed: 4 May 2020). Braun, V. and Clarke, V. (2006) ‘Using thematic analysis in psychology.’ Qualitative research in psychology, 3(2), 77-101. Available at: https://www.tandfonline.com/doi/abs/10.1191/1478088706qp063oa (Accessed: 6 May 2020). Clarke, V. and Braun, V. (2013) Successful Qualitative Research: A Practical Guide for Beginners. London: SAGE. ISBN. 9781847875815 Chin, C.K., Munip, H., Miyadera, R., Thoe, N.K., Ch’ng, Y.S. and Promsing, N. (2019) ‘Promoting Education for Sustainable Development in Teacher Education integrating Blended Learning and Digital Tools: An Evaluation with Exemplary Cases.’ Eurasia Journal of Mathematics, Science and Technology Education, 15(1), em1653. Available at: https://doi.org/10.29333/ejmste/99513 (Accessed: 1 May 2020). Compton, M. and Allen, J. (2018) ‘Student Response Systems: a rationale for their use and a comparison of some cloud-based tools.’ Compass: Journal of Learning and Teaching, (11)1, 2018. Available at: https://journals.gre.ac.uk/index.php/compass/article/view/696 (Accessed: 2 November 2020). Covill, A.E. (2011) ‘College students’ perceptions of the traditional lecture method.’ College Student Journal, 45(1), 92-102. Available at: https://eric.ed.gov/?id=EJ996351 (Accessed: 8 July 2020). Crouch, C.H and Mazur, E. (2001) ‘Peer instruction: Ten years of experience and results.’ American Journal of Physics, 69(9), 970-977. Available at: https://doi.org/10.1119/1.1374249 (Accessed: 8 July 2020). Detyna, M. and Kadiri, M. (2019) ‘Virtual reality in the HE classroom: feasibility and the potential to embed in the curriculum.’ Journal of Geography in Higher Education, (44)3, 474. Available at: https://doi.org/10.1080/03098265.2019.1700486 (Accessed: 12 January 2020). DiPiro, J.T. (2009) ‘Why do we still lecture?’ American Journal of Pharmaceutical Education, 73(8), 137. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2828296/ (Accessed: 4 February 2020). Dunbar, L. (2017) ‘Using Padlet to increase student interaction with music concepts.’ General Music Today, 30(3), 26-29. Available at: https://doi.org/10.1177/1048371316687076 (Accessed: 6 March 2020). Eick, C.J. and King Jr, D.T. (2012) ‘Nonscience majors’ perceptions on the use of YouTube video to support learning in an integrated science lecture.’ Journal https://journals.gre.ac.uk/index.php/compass/article/view/714 https://www.tandfonline.com/doi/abs/10.1191/1478088706qp063oa https://doi.org/10.29333/ejmste/99513 https://journals.gre.ac.uk/index.php/compass/article/view/696 https://eric.ed.gov/?id=EJ996351 https://doi.org/10.1119/1.1374249 https://doi.org/10.1080/03098265.2019.1700486 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2828296/ https://doi.org/10.1177/1048371316687076 Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 16 of College Science Teaching, 42(1), 26. Available at: https://eric.ed.gov/?id=EJ988876 (Accessed: 5 March 2020). Feldon, D.F. (2010) ‘Why magic bullets don’t work.’ Change: The Magazine of Higher Learning, 42(2), 15-21. Available at: https://doi.org/10.1080/00091380903563043 (Accessed: 7 July 2020). Fiester, H. and Green, T. (2016) ‘Student use of backchannels.’ TechTrends, 60(4), 404-408. Available at: https://link.springer.com/article/10.1007/s11528-016-0069-9 (Accessed: 9 March 2020). Fitzgerald, R. and Henderson-Martin, H. (2015) ‘Transforming the first year experience (HE) with digital literacy via techno-social engagement and evaluation.’ In: Jefferies, A. and Cubric, M. (eds.) Proceedings of the 14th European Conference on e-Learning, ECEL 2015, University of Hertfordshire, Hatfield, UK, 29-30 October 2015. Reading, UK: Academic Conferences and Publishing International Limited. 2048-8645, 200-206. ISBN. 9781910810705 Freeman, S., Eddy, S.L., McDonough, M., Smith, M.K., Okoroafor, N., Jordt, H. and Wenderoth, M.P. (2014) ‘Active learning increases student performance in science, engineering and mathematics.’ Proceedings of the National Academy of Sciences, 111(23), 8410-8415. Available at: https://doi.org/10.1073/pnas.1319030111 (Accessed: 1 December 2019). Fried, C.B. (2008) ‘In-class laptop use and its effects on student learning.’ Computers and Education, 50(3), 906–914. Available at: https://doi.org/10.1016/j.compedu.2006.09.006 (Accessed: 20 January 2020). Fugard, A.J. and Potts, H.W. (2015) ‘Supporting thinking on sample sizes for thematic analyses: a quantitative tool.’ International Journal of Social Research Methodology, 18(6), 669-684. Available at: https://doi.org/10.1080/13645579.2015.1005453 (Accessed: 12 December 2019). Gehlen-Baum, V., Weinberger, A., Pohl, A. and Bry, F. (2014) ‘Technology use in lectures to enhance students’ attention’. Paper presented at the European Conference on Technology Enhanced Learning. Available at: https://link.springer.com/chapter/10.1007/978-3-319- 11200-8_10 (Accessed: 2 November 2020). Gill, J., Harrison, B., Wood, T.J., Ramnanan, C.J. and Jalali, A. (2014) ‘Investigating the Use of Social Networking Tools Among Medical Students.’ Education in Medicine Journal, 6(4). Available at: https://www.researchgate.net/publication/279514916 (Accessed: 4 November 2020). Gross-Loh, C. (2016) ‘Should colleges really eliminate the college lecture?’ The Atlantic. July 14, 2016. Available at: https://www.theatlantic.com/education/archive/2016/07/eliminating- the-lecture/491135/ (Accessed: 5 November 2020). Hall, R.H., Collier, H.L., Thomas, M.L. and Hilgers, M.G. (2005) ‘A student response system for increasing engagement, motivation and learning in high enrollment lectures.’ AMCIS https://eric.ed.gov/?id=EJ988876 https://doi.org/10.1080/00091380903563043 https://link.springer.com/article/10.1007/s11528-016-0069-9 https://doi.org/10.1073/pnas.1319030111 https://psycnet.apa.org/doi/10.1016/j.compedu.2006.09.006 https://doi.org/10.1080/13645579.2015.1005453 https://link.springer.com/chapter/10.1007/978-3-319-11200-8_10 https://link.springer.com/chapter/10.1007/978-3-319-11200-8_10 https://www.theatlantic.com/education/archive/2016/07/eliminating-the-lecture/491135/ https://www.theatlantic.com/education/archive/2016/07/eliminating-the-lecture/491135/ Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 17 2005 Proceedings, 255. Available at: https://www.researchgate.net/publication/220893650 (Accessed: 2 November 2020). Huijser, H. and Kek, M.Y.C.A. (2016) ‘PBL and technology-supported learning: Exploring the right blend.’ In: Problem-Based Learning: Perspectives, Methods and Challenges, Hauppauge, New York: Nova Science Publishers, Inc., 149-164. ISBN. 9781634851831 Mishra, P. and Koehler, M.J. (2006) ‘Technological Pedagogical Content Knowledge: A Framework for Teacher Knowledge.’ Teachers College Record, 108(6), 1017-1054. Available at: https://doi.org/10.1111/j.1467-9620.2006.00684.x (Accessed: 11 October 2019). Kaleta, R. and Joosten, T. (2007) ‘Student response systems.’ A University of Wisconsin System Study. Research Bulletin, 2007(10) Available at: http://www.westfield.ma.edu/cit/wp- content/uploads/Study-of-Clickers.pdf (Accessed: 2 November 2020). Kappers, W.M. and Cutler, S.L. (2015) ‘Poll Everywhere! Even in the Classroom: An Investigation into the Impact of Using PollEverwhere in a Large-Lecture Classroom.’ Computers in Education Journal, 6(20), 21. Available at: https://core.ac.uk/download/pdf/217173455.pdf (Accessed: 4 November 2020). Kelly, P.A., Haidet, P., Schneider, V., Searle, N., Seidel, C.L. and Richards, B.F. (2005) ‘A comparison of in-class learner engagement across lecture, problem-based learning and team learning using the STROBE classroom observation tool.’ Teaching and learning in medicine, 17(2), 112-118. Available at: https://pubmed.ncbi.nlm.nih.gov/15833720/ (Accessed: 4 November 2020). Koehler, M.J. and Mishra, P. (2009) ‘What is technological pedagogical content knowledge?’ Contemporary Issues in Technology and Teacher Education 9(1), 60-70. Available at: https://doi.org/10.1177/002205741319300303 (Accessed: 3 November 2020). King, D.B. and Joshi, S. (2006) Quantitative measures of personal response device effectiveness. Philadelphia: Drexel University Libraries. Available at: https://www.researchgate.net/publication/28674625 (Accessed: 7 March 2020). Kirschner, P.A. and Karpinski, A.C. (2010) ‘Facebook and academic performance.’ Computers in human behavior, 26(6), 1237-1245. Available at: https://www.sciencedirect.com/science/article/abs/pii/S0747563210000646 (Accessed: 9 March 2020). Licorish, S.A., Owen, H.E., Daniel, B. and George, J. (2018) ‘Students’ perception of Kahoot!’s influence on teaching and learning.’ Research and Practice in Technology Enhanced Learning, 13(9). Available at: https://doi.org/10.1186/s41039-018-0078-8 (Accessed: 7 March 2020). Lin, Y.-C., Liu, T.-C. and Chu, C.-C. (2011) ‘Implementing clickers to assist learning in science lectures: The Clicker-Assisted Conceptual Change model.’ Australasian Journal of Educational Technology, 27(6) Available at: https://doi.org/10.14742/ajet.924 (Accessed: 4 December 2019). https://psycnet.apa.org/doi/10.1111/j.1467-9620.2006.00684.x http://www.westfield.ma.edu/cit/wp-content/uploads/Study-of-Clickers.pdf http://www.westfield.ma.edu/cit/wp-content/uploads/Study-of-Clickers.pdf https://core.ac.uk/download/pdf/217173455.pdf https://pubmed.ncbi.nlm.nih.gov/15833720/ https://doi.org/10.1177%2F002205741319300303 https://www.researchgate.net/publication/28674625 https://www.sciencedirect.com/science/article/abs/pii/S0747563210000646 https://doi.org/10.1186/s41039-018-0078-8 https://doi.org/10.14742/ajet.924 Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 18 Lyubartseva, G. (2013) ‘Influence of Audience Response System Technology on Student Performance in Organic Chemistry Lecture Class.’ Education, 133(4), 439-443. Available at: https://www.learntechlib.org/p/155372/ (Accessed: 3 November 2020). Mallin, I. (2017) ‘Lecture and active learning as a dialectical tension.’ Communication Education, 66(2), 242-243. Available at: https://doi.org/10.1080/03634523.2016.1275720 (Accessed: 3 November 2020). Mann, S. and Robinson, A. (2009) ‘Boredom in the lecture theatre: An investigation into the contributors, moderators and outcomes of boredom amongst university students.’ British Educational Research Journal, 35(2), 243-258. Available at: https://doi.org/10.1080/01411920802042911 (Accessed: 2 November 2020). Mitra, B., Lewin‐Jones, J., Barrett, H. and Williamson, S. (2010) ‘The use of video to enable deep learning.’ Research in Post‐Compulsory Education, 15(4), 405-414. Available at: http://dx.doi.org/10.1080/13596748.2010.526802 (Accessed: 3 October 2020). Mays, N. and Pope, C. (1995) ‘Rigor and qualitative research.’ British Medical Journal 311(1), 109-12. Available at: https://doi.org/10.1136/bmj.311.6997.109 (Accessed: 4 October 2020). Pollock, S.J. (2005) ‘No single cause: learning gains, student attitudes and the impacts of multiple effective reforms.’ 790(1), 137. Paper presented at the AIP Conference Proceedings. Available at: https://doi.org/10.1063/1.2084720 (Accessed: 2 November 2020). Poulis, J., Massen, C., Robens, E. and Gilbert, M. (1998) ‘Physics lecturing with audience paced feedback.’ American Journal of Physics, 66(5), 439-441. Available at: https://doi.org/10.1119/1.18883 (Accessed: 2 November 2020). Rowe, S. (2004) ‘A case for virtual guest lecturers: the experience of using practitioners in asynchronous discussion forums with an undergraduate auditing class.’ Available at: https://www.researchgate.net/publication/49403595 (Accessed: 8 September 2020). Saxena, S.K., Kumar, S., Maurya, V.K., Sharma, R., Dandu, H.R. and Bhatt, M. (2020) ‘Current Insight into the Novel Coronavirus Disease 2019 (COVID-19).’ Epidemiology, Pathogenesis, Diagnosis and Therapeutics’, 1-8. Available at: https://doi.org/10.1007/978- 981-15-4814-7_1 (Accessed: 4 November 2020). Salmon, G. (2014) ‘Learning Innovation: A Framework For Transformation.’ European Journal of Open, Distance and e-Learning, 17(2),220. Available at: https://doi.org/10.2478/eurodl-2014-0031 (Accessed: 2 November 2020). Schmidt, H.G., Wagener, S.L., Smeets, G.A., Keemink, L.M. and van der Molen, H.T. (2015) ‘On the use and misuse of lectures in higher education.’ Health Professions Education, 1(1), 12-18. Available at: https://cyberleninka.org/article/n/623972.pdf (Accessed: 6 November 2020). https://www.learntechlib.org/p/155372/ https://doi.org/10.1080/03634523.2016.1275720 https://doi.org/10.1080/01411920802042911 http://dx.doi.org/10.1080/13596748.2010.526802 https://doi.org/10.1136/bmj.311.6997.109 https://doi.org/10.1063/1.2084720 https://doi.org/10.1119/1.18883 https://www.researchgate.net/publication/49403595 https://doi.org/10.1007/978-981-15-4814-7_1 https://doi.org/10.1007/978-981-15-4814-7_1 https://doi.org/10.2478/eurodl-2014-0031 https://cyberleninka.org/article/n/623972.pdf Articles Compass: Journal of Learning and Teaching, Vol 14, No 1, 2021 19 Shon, H. and Smith, L. (2011) ‘A review of Poll Everywhere audience response system.’ Journal of Technology in Human Services, 29(3), 236-245. Available at: https://doi.org/10.1080/15228835.2011.616475 (Accessed: 4 November 2020). Stacy, J. (2009) ‘The guide on the stage: In defense of good lecturing in the history classroom.’ Social Education, 73(6), 275-278. Available at: https://www.socialstudies.org/social-education/73/6/guide-stage-defense-good-lecturing- history-classroom (Accessed: 4 November 2020). Sun, L., Tang, Y. and Zuo, W. (2020) ‘Coronavirus pushes education online.’ Nature Materials 19(1) 687. Available at: https://doi.org/10.1038/s41563-020-0678-8 (Accessed: 4 November 2020). Stojšić I., Ivkov-Džigurski A. and Maričić O. (2019) ‘Virtual Reality as a Learning Tool: How and Where to Start with Immersive Teaching.’ In: Daniela, L. (ed.) Didactics of Smart Pedagogy. Cham, Switzerland: Springer. ISBN. 3030015505 Sundt, M (2019) ‘Professors, Ask Hard Questions of Your Online Providers.’ Inside Higher Ed; January 30, 2019. Available at: https://www.insidehighered.com/digital- learning/views/2019/01/30/professors-should-ask-hard-questions-their-corporate-online (Accessed: 5 September 2020). Titsworth, B.S. (2001) ‘Immediate and delayed effects of interest cues and engagement cues on students’ affective learning.’ Communication Studies, 52(3), 169-179. Available at: https://www.tandfonline.com/doi/abs/10.1080/10510970109388552 (Accessed: 6 November 2020). Van Dijk, L., Van Der Berg, G. and Van Keulen, H. (2001) ‘Interactive lectures in engineering education.’ European Journal of Engineering Education, 26(1), 15-28. Available at: https://doi.org/10.1080/03043790123124 (Accessed: 6 November 2020). van Hoek, R., Godsell, J. and Harrison, A. (2011) ‘Embedding “insights from industry” in supply chain programmes: the role of guest lecturers.’ Supply Chain Management: An International Journal, 16(2), 142-147. Available at: https://doi.org/10.1108/13598541111115383 (Accessed: 6 November 2020). Wang, A.I. (2015) The wear out effect of a game-based student response system. Computers and Education, 82(1), 217-227. Available at: https://doi.org/10.1016/j.compedu.2014.11.004 (Accessed: 2 November 2020). https://doi.org/10.1080/15228835.2011.616475 https://www.socialstudies.org/social-education/73/6/guide-stage-defense-good-lecturing-history-classroom https://www.socialstudies.org/social-education/73/6/guide-stage-defense-good-lecturing-history-classroom https://doi.org/10.1038/s41563-020-0678-8 https://www.insidehighered.com/digital-learning/views/2019/01/30/professors-should-ask-hard-questions-their-corporate-online https://www.insidehighered.com/digital-learning/views/2019/01/30/professors-should-ask-hard-questions-their-corporate-online https://www.tandfonline.com/doi/abs/10.1080/10510970109388552 https://doi.org/10.1080/03043790123124 https://doi.org/10.1108/13598541111115383 https://doi.org/10.1016/j.compedu.2014.11.004