ALT-J, Research in Learning Technology Vol. 15, No. 1, March 2007, pp. 3–20 ISSN 0968-7769 (print)/ISSN 1741-1629 (online)/07/010003–18 © 2007 Association for Learning Technology DOI: 10.1080/09687760600837090 The use of electronic voting systems in lectures within business and marketing: a case study of their impact on student learning George Masikunas, Andreas Panayiotidis and Linda Burke* Kingston University, UK Taylor and Francis LtdCALT_A_183646.sgm10.1080/09687760600837090ALT-J Research in Learning Technology0968-7769 (print)/1741-1629 (online)Original Article2006Taylor & Francis143000000September 2006LindaBurkeLinda.Burke@kingston.ac.uk This article presents a case study of the impact on student learning of introducing an electronic voting system (EVS) into large-group lectures for first-year undergraduate students undertaking degrees in marketing and business systems. We discuss the potential for using EVS-style interactive lectures in marketing and business programmes. We then describe how we planned the session and selected and implemented the EVS system. We go on to present an evaluative research project, which was undertaken on the innovation using case-study methodology, and assess its impact on student learning. Data for the evaluation were collected through questionnaire and focus groups with a sample of students. The data were analysed using thematic analysis. The findings show how students perceived the use of EVS in large lectures and how their learning was affected. A ‘three- fold typology’ emerged that explains how students related to the EVS and how their perceptions of EVS changed over time. The discussion links these findings to the literature on different paradigms of learning and teaching, using Renshaw’s framework, and examines how the EVS-style lectures promote deep and active learning within the constructivist, social constructivist and metacognitive learning paradigms identified in Renshaw’s model. The conclusions show how the use of a user- friendly EVS in large lectures motivates students, develops students’ cognitive and social learning skills, and improves learning effectiveness. Introduction Kingston University provides undergraduate degrees in business and marketing. A large numbers of students (500) undertake certain Year One modules at the same time, which have traditionally been delivered through lectures. A decision was taken to test the use of an electronic voting system (EVS)-style interactive lecture and * Corresponding author. Faculty of Health and Social Care Sciences, Kingston University and St. George’s, Sir Frank Lampl Building, Kingston Hill Campus, Kingston-upon-Thames KT27 7LB, UK. 4 G. Masikunas et al. PowerPointVote System (PPVOTE) in first-year marketing and business studies modules. This article begins with background information on how we proposed to use EVS- style interactive lectures to address issues raised by student evaluations. We then present the potential benefits of using EVS-style lectures, how we believed we could make best use of EVS as a tool for improving student learning and how the literature on EVS supported our thinking. We go on to discuss how we prepared for and implemented the interactive EVS-style lectures and the associated evaluative research to measure the impact on student learning and achievement. In most higher education institutions the lecture is the widely accepted method of knowledge transmission (Jarvis, 2002); however, research shows that the traditional style of lecture is the least-favoured method of instruction (Butler, 1992; Bligh, 2000). Bligh (2000) suggests that lectures are mostly useful for imparting information but not for developing powerful minds. This echoes Boyer (1990), who suggested that: Great teachers create a common ground of intellectual commitment. They stimulate active, not passive, learning and encourage students to be critical, creative thinkers, with the capacity to go on learning … (p. 24) As a teaching team we became aware of a range of problems with traditional lectures for first-year undergraduate students in marketing and business studies. Our students were telling us that the traditional lectures were ‘a bit of a turn off’. Internal feedback from our students led us to realise that our traditional lectures were not serving their purpose. Several problems with the traditional face-to-face lecture format were identified from both students and lecturers, particularly with large classes: ● a general lack of student engagement during lectures, both with the material presented, and with their peers; ● large classes typically do not allow students an opportunity to present their opinions and to receive immediate feedback; and ● there was little opportunity for lecturers to test students’ understanding of material before moving on to new material. We undertook an analysis of our traditional lectures using Bloom’s Taxonomy of the Cognitive Domain (Bloom, 1956). Our students gave high scores to our lectures for Bloom’s lower-order cognitive skills (knowledge and comprehension) and low scores for higher-order cognitive skills (application, analysis, synthesis and evaluation). Our intention was that a new-style interactive lecture would score reasonably high for both sets of cognitive skills. The potential for using EVS-style interactive lectures We decided that, by using EVS as the basis for interactive lectures, we wanted to achieve a number of things in terms of enhancing student learning. We wanted to: EVS in lectures within business and marketing 5 ● create a stimulating learning environment in the lecture room that would engage students through making them active rather than passive learners; ● enhance students’ overall performance in the Year One modules; ● reduce the number of assessment retakes; ● improve retention rates; ● build self-confidence in the students through encouraging participation; and ● stimulate student–student interactions and student–teacher interaction in a way that led to a shared process of knowledge-building. Mazur (1997) and Dufresne et al. (1996) implemented EVS into interactive lectures with the aim of developing students’ cognitive skills. Mazur (1997) also wished to foster collaborative learning through integrating peer–student instruction as an integral element of an EVS-style interactive lecture. Others (for example, Eliot, 2001) suggested that using EVS in lectures is appropriate in subject disciplines that are perceived to be particularly challenging to student learning; for example, physical sciences, engineering, computing and statistics: … I was aware that it was a course that students have historically found challenging. Further, in a lecture environment the students may be unwilling to volunteer information regarding their level of understanding of the material covered. (Eliot, 2001, p. 80) Boyle and Nichol (2003) said that: The aim [of introducing EVS-style interactive lectures] was to aid students’ transition into higher education, improve retention, give a better first year experience and introduce active learning and discussion concepts into large classes. (p. 43) Lui et al. (2003) used EVS to overcome the communicative limitations that large lectures traditionally create and to enable students to construct deeper meaning in terms of the content presented in lectures. Others (for example, Horowitz, 1988; Hake, 1996; Abrahamson, 1999) adopted student response systems in lectures with the aim of improving learners’ attention or increasing assessment scores, or to facilitate interaction between students in the lecture (Sarah, 2002). Hake (1996) concluded that interactive engagement in student learning was preferable to traditional classroom teaching and: … strongly suggests that the classroom use of IE [interactive engagement] methods can increase mechanics-course effectiveness well beyond that obtained by traditional practice. (p. 72) Laurillard (1993) presented a persuasive argument for interactive lectures in terms of the essential role that feedback plays in student learning. She suggested that action on the part of the learner without some sort of feedback is unproductive and that receiving meaningful and useful feedback is important for student learning. Plan for using EVS in our marketing and business modules We proposed to use the EVS-style lecture to: 6 G. Masikunas et al. ● deliver interactive lectures that incorporated information giving, small-group learning to answer specific questions/complete group learning tasks, and group decision-making; ● use an EVS that enabled students to communicate their answers to the group activ- ity and instantly converted students’ responses (in the form of a histogram projected onto a screen at the front of the lecture room); ● use this process to provide formative feedback to all students through which they could judge their own responses against those of others and against the correct response provided by the teacher; and ● improve student concentration in lectures, promote deeper approaches to learning and enable students to develop a wider range of cognitive skills (Bloom, 1956; Renshaw, 1995). In drawing up our plans we were mindful of some barriers to its success, including potential concerns by students that less material might be covered in interactive EVS- style lectures, a concern that students might focus more on achieving better marks and that students might use the group-activity time in the lecture to socialise rather than complete the activity. Selection of the EVS system The EVS system we selected needed to be congruent with promoting different types of learning and the aims we identified for this project. We wanted interactive EVS-style lectures that would appeal to all types of learner styles, but that would especially promote the ‘Deep Active’ type (Marton, 1994). We were mindful, too, of work by Brown and Brailsford (2004), who suggested that learners are adapters whose learning styles can change according to the learning environment. The EVS system we selected would change the learning environment of lectures and needed to help us to bring together interactivity, group problem- solving following freshly taught material, and peer teaching and feedback to students. Finally, we also took into account the work of other pioneers in the use of EVS systems in teaching and learning (Horowitz, 1988; Dufresne et al., 1996; Mazur, 1997; Crouch & Mazur, 2001; Liu et al., 2003; Draper & Brown, 2004). Introduction of the innovation After comparing different EVS systems we elected to use the Personal Response System supplied by Educue for our mass-attended classes. The systems purchase did require certain additional hardware and software conditions to be met. These were satisfied by the information technology provision in the lecture theatres across the university. There is a university-wide network linked to a proprietary virtual learning system, which feeds into classrooms where a data-projection system is run from a stand-alone computer linked to large screens or electronic white boards. This infor- mation-technology-enriched classroom met the hardware and software pre-requisite. EVS in lectures within business and marketing 7 Absence of this would have constituted a major barrier to adoption of the EVS-style lecture. Preparation for the innovation Preparation involved a variety of events, including gaining funding approval, purchas- ing, staff training in the use of EVS, practice sessions in our lecture halls using EVS, and embedding dedicated EVS-focused slides (with group work questions and possi- ble answers) into the PowerPoint presentation. Practice sessions allowed us to load the special EVS-dedicated software onto the computer in the lecture hall, ensuring that the sensor device could pick up responses from the remote controls used to signal student choice from the answers offered. The commitment involved and timing of lectures became significant elements of the plans for implementing this innovation. Time was needed at the beginning of each lecture to set up the EVS system in the lecture hall, distribute remote control panels, allow students to form small teams of three to five, undertake the integrated EVS-type question and answer sessions, and ensure that the equipment was taken away at the end of the lecture session. We will come back to discuss how the use of time in the lecture changed as a result of moving to EVS-style interactive lectures. Implementation of the innovation Suggestions for sequencing of learning activities in an interactive EVS-style lecture vary in the literature. Mazur (1997) and Dufresne et al. (1996) propose different sequences in an EVS-style interactive lecture. The major difference between the two lies in the extent to which it is the individual student who answers the question/ problem posed, or the small group who agrees an answer. Mazur (1997) favoured an individual-student approach, while Dufresne et al. (1996) proposed a group-wide sequence for providing the answers. Nicol and Boyle (2003) reported that students in one university found Mazur’s peer instruction sequence to be more helpful to their learning in preference to the classroom-wide sequence advocated by Dufresne et al. (1996). The two lecturers involved in this project created similar sequences, both of which are based on Dufesne et al. (1996), but with some minor adaptations. Figure 1 illustrates how EVS operates within the following sequence of events in an interactive lecture. Figure 1. The EVS learning cycle and sequence The EVS involves distribution of remote handsets and dividing the large group into small work groups of three to five students. The lecturer then introduces a new piece of module content that has a question/learning task posed, and a set of possible answers. Each small group discusses the question/completes the task and, through collaboration, interactivity, group decision-making and discussion, agrees their answer as a group. In this process, students are encouraged to be involved in peer student teaching where individual group members explain their reasoning for selecting a given answer. As part of the discussion, the group evaluates each possible answer from the choices, before agreeing an answer. 8 G. Masikunas et al. Group answers are submitted by pressing the appropriate button on the remote handset, which is picked up by sensors linked to a computer that projects the results of all the responses. The voting process takes a very short time. A histogram is generated that displays on a screen in the front to the lecture room the overall outcome choices. At this point the lecturer can do a number of things: ● Indicate the correct answer and give the reasoning to support it, followed by a classroom-wide discussion allowing team members to support/explain their choice. ● Follow-on discussion, but without giving the correct answer, repeat the question allowing every team to answer again, permitting statistical comparison on any changes in voting. ● Give the correct answer, with reasoning, followed by a summary. The lecture continues with new taught material, followed by a second EVS sequence. The sequence presented in Figure 1 can be repeated a number of times during the lecture. EVS was first used in the lectures presented by the module leader for the Business Information Systems module. Approximately 240 students undertook this ��� ������ �� ��� ����� ����� � �� ����� �� ���������� � �� �� �� �� ����� �� ��� � ���� ������ ���� � �� �� ��� � ��� �������� �� �� �� ��� ��������� ���� ��� �� � ���� ������ ����� � �� ���� ��� � �� � � � ���� �� �� ��� ��� � �������� � ��� !�� ���� � ���� � ��� ���������� "���� �� ��� Figure 1. The EVS learning cycle and sequence EVS in lectures within business and marketing 9 module together. Other lecturers observed these sessions and shortly afterwards the EVS-style lecture was adopted for the remainder of Year One modules undertaken by the same students. The features that impressed the other lecturers, and that led them to adopt this approach, included the students’ response to this new style of lecturing and the way students engaged with each other, with the lecturers and with the course content, during the EVS-style lectures. When we implemented the EVS-style lecture we became aware that use of the lecture time changed with EVS-style lectures. Figure 2 shows the typical time alloca- tion used in our traditional-style lectures as compared with the use of lecture time in EVS-interactive style lectures. Figure 2. Typical time allocation for traditional lectures compared with EVS-interactive style lectures The evaluation framework Our research objectives with respect to student learning were, firstly, to measure students’ perceptions about the use of EVS in large-group lectures, and, secondly, to identify and explore the specific factors that increased the effectiveness of learning in an interactive lecture using EVS. Additionally, we wished to examine students’ atti- tudes and feelings about EVS lectures. We also wanted to examine the extent to which quantitative data (achievement and progression) correlated with qualitative data. A decision was taken to undertake an evaluative research project using case study design (Alvarez et al., 1990; Brearley, 1993; Greenwald, 1991; Yin, 1994; Carney, 1995; Tellis, 1997). Selecting an evaluative research framework is important to ensure that an innovation is subject to rigorous scrutiny and measures the extent to which success criteria or proposed outcomes are achieved. This assures that educational ends precede technological means. Data collection and analysis A case study as an evaluative research design lends itself to using a number of data collection methods. We focused the evaluative case study on measuring the impact of EVS lecture time utilisation Introduction 10% Conclussion 10% Interaction 36% Lecture 44% Traditional lecture time utilisation Introduction 6% Lecture 68% Interaction 20% Conclusion 6% Figure 2. Typical time allocation for traditional lectures compared with EVS-interactive style lectures 10 G. Masikunas et al. EVS-style lectures on student learning and achievement. Data were collected using survey questionnaires, focus groups and quantitative data related to assessment results. We undertook a survey of students that involved the distribution of a questionnaire to those students who attended the Marketing and the Business Information Systems modules. These survey questionnaires were distributed to students in week six of their modules. We also undertook focus group interviews with a sample of students to record and analyse their ideas, feelings, attitudes and perceptions about their EVS experience and its impact on their learning. Quantitative data about assessment results were obtained from minutes of examination board meetings. The survey questionnaire yielded both qualitative and quantitative data. The questionnaires and focus groups were analysed using thematic analysis (Aronson, 1994). Thematic analysis involved identifying patterns in the data and classifying data according to these, finding themes from within these and then determining which of these themes were key themes; that is, those that form a comprehensive picture of the collective experience of the stakeholders. In addition to thematic analysis of the qualitative data, we used the quantitative data about student performance in assessments to ascertain any differences in assessment marks for these students, compared with previous first-year students who undertook these modules through traditional lectures. Findings The response rate for the survey questionnaire was 40% (n = 165). Analysis of the survey questionnaires indicated that students perceived the effectiveness of their learning was improved through the EVS lectures (mean response rate, 4.25 out of 5). Thematic analysis of the survey questionnaires enabled us to identify the following themes that contributed to this improved learning: ● Enjoyment of the EVS-style lectures. ● Consultation/participation. ● Reinforcement feedback. ● Team decision-making. ● Student interactivity. ● Student perceptions of the role of lectures. Data from the focus groups confirmed that these were the key themes related to student learning and achievement. Each of these themes are now discussed in turn. Enjoyment of the EVS-style lectures was a recurring theme from the survey question- naires and focus groups. The majority of students expressed a high degree of satisfac- tion with the EVS-style lectures. Analysis of the survey questionnaires gave an aggregate score of 4.7 out of 5 for the question on enjoyment. Lectures have come alive. Before, in most classes, I think most students were not listening. Some people sitting next to you did not want to know. EVS in lectures within business and marketing 11 Consultation/ participation was identified as a factor in improving learning, with a mean score of 4.35 out of 5. Three types of consultation/participation emerged from the focus groups as having a positive impact on learning. Students having the opportunity: ● to be active in the large group lecture; ● to discuss problems with others in their small group; and ● to discuss issues with the lecturer within the lecture, thus opening up a new channel of communication between lecturer and student and instigating a change in the student–lecturer relationship. Reinforcement feedback as part of the learning process during the EVS lectures emerged as another theme (mean score, 4.45). In one of several responses about reinforcement and feedback, a student commented that when the lecturer gave the correct answer after the EVS activity, it was an extension of: … revising my understanding and … a function of learning reinforcement. What this seemed to say was that feedback and reinforcement from lecturers helped this student to re-think his understanding of the content and reinforced learning. Students viewed the EVS-style lecture as a means of providing them with feedback about newly taught material, and some came to see this as confirming or challenging the meaning they had constructed and derived. Where their response was ‘correct’ they saw this as providing positive reinforcement, whereas ‘wrong responses’ helped to clarify a taught concept so that their state of knowledge was ‘on track’, which counts as another form of ‘reinforcement’. Feedback and reinforcement within the EVS lectures was very important to most students, and helped to develop their confidence, as the following quotes illustrate: Further discussion or re-explanation was given to bridge the learning gap. It motivates to get feedback about what you learned. It gives us confidence, if the answer is right … and if you answer wrong you will see the correct answer later on and there’s a bigger chance for you to remember that answer. Team decision-making was another emerging theme (mean score, 4.6) that had a positive impact on students’ perceptions of their learning. A number of reasons for this were identified by students in the questionnaires and focus groups. Team decision-making gave students opportunities to look at different options for problem- solving and decision-making. … you get to know more about the other options … … you would think that this option would be right, but you think on the other option as well … Team decision-making was seen as positive because of the opportunities it gave students to find out why each student believed certain answers to be correct. … you can ask a person in your group who has given the ‘A’ answer, ‘How do you come to the answer A’ … 12 G. Masikunas et al. Another recurring response was the positive effect of learning from others in groups (peer instruction). A third recurring response was that of developing self-confidence through joint decision-making. It [joint decision-making] builds your confidence. A small minority of students said that they would have preferred to have been given individual handsets so that they could work through the questions on their own rather than within a small group. This response tended to come from the earliest focus group. The issue of some students preferring to work independently rather than as part of a team did not appear in later focus groups. This may be due to the fact the early focus group took place prior to the first module assessment and assessment results from the first module demonstrated an improvement in results from previous years. In addition, if students were to tackle the problems posted in the EVS lecture on their own, rather than in a group, the element of peer learning would be lost. The perceptions of the majority of students in this study were that peer learning was valuable and enhanced their individual learning. Interactivity and the positive impact on learning that interactivity within the EVS- style lecture created was a theme that emerged from the survey questionnaires (mean score, 4.7). An overwhelming majority of students appreciated having the time to work in small groups during the large lecture, and preferred group working and group decision-making. Most students felt that having the opportunity to interact in the lecture enhanced the learning experience. It was completely different from the way you assumed the lecture was going to be like. There was a group vibe; everyone conferred and you got to be in your group. There was a question in front of you and everyone had a discussion. Students reported that the interactivity in class spilled over into their interactions with fellow students after the class was over. Some students reported, however, that when other students did not take the group discussion/team decision-making time seriously, this detracted from their learning. Students’ perception of the value of the role of lectures emerged as a theme from the focus groups particularly. The majority favourably compared the EVS lecture with traditional lectures in terms of enhancing their learning effectiveness. Analysis of the data, however, showed a very complex picture about how students perceive that their learning takes place. We are really learning more. We can really learn with our group. I do not have to spend ages re-reading my lecture notes to get it. … felt I really understood the basics; the concepts of marketing. Before, I was not so sure. … if it [EVS] helps you to know, then learning this way is good. In addition, the value of learning from fellow-students emerged from the data. You can join up with your friends in small groups. You learn from your mates. EVS in lectures within business and marketing 13 However, a very small number of students expressed concern about optimum use of lecture time. … worried that we will not cover everything in time for the exams and we will get lower grades even though we enjoyed it. These students also expressed concern about the amount of ‘dead time’ in the EVS lecture (e.g. time spent handing out and collecting in the remote voting devices). This appears to indicate a deep concern on the part of a small number of students about their overall performance in a module. Those who expressed concerns related to optimum use of lecture time may have been demonstrating that their discomfort of new approaches to learning could adversely affect their level of achievement. In terms of assessment results, we found that student average aggregate performance improved for both modules in 2003/04 compared with the previous cohort of students. The student profile was similar across both cohorts and the same module leaders gave the lectures in both years. The variable was that in the 2003/04 cohort the lectures were interactive using EVS. For both modules the average aggregate student marks rose, from 56% in the 2002/03 first-year cohort to 59% in the 2003/04 cohort. Discussion From the data analysis we suggest that there may be a model emerging that depicts three stages experienced by students arising from a sequence of EVS-style lectures. Stage 1: early mass acceptance This occurred in the early period when EVS was launched and where there was immediate and widespread acceptance. The vast majority of students described their initial experiences of EVS in large-group lectures as highly satisfactory, and this may reflect their engagement with the new. There might also have been a halo effect arising from lecturer enthusiasm for EVS. Students made initial comparisons between the traditional lectures and EVS lectures, appreciating the new benefits delivered by EVS. The features that struck them as special to EVS included interactivity, engagement, group and peer instruc- tion, rapid feedback of student responses and better understanding. Their attitudes and feelings towards the use of EVS were strongly held and felt. In stage 1 the novelty factor is pervasive, and students express that this style of lecture relieves the monotony of traditional lectures. Stage 2: reflection This occurs when students start a deeper reassessment of their newer blended learning environment created through the EVS lecture. They clearly begin to articu- late the difference between traditional and EVS-style lectures and the effect that both have on their learning. They also begin to assess and analyse the strengths and 14 G. Masikunas et al. weaknesses of both types of lectures. For some students, reservations begin to appear. EVS lectures are still seen as enjoyable but some students begin to perceive distrac- tions when other students do not take the group discussion time seriously. The euphoria of the first stage passes and students demonstrate a more reflective view in relation to EVS as a process. As one student stated: The novelty wore off, but I still liked it. From our findings, students start to fall into a number of clusters according to their ideas, feelings, perceptions and attitudes: ● Type 1: ‘Must have it’. ● Type 2: ‘Need it but with minor reservations’. ● Type 3: ‘Sceptical about it but it needs to improve’. Type 1 students are those who liked this way of learning. Type 2 and type 3 students demonstrated more reflective and divergent thinking about EVS. Our focus groups indicated that students who had very fixed ideas about the traditional lecture, and who came from educational systems where there was an extremely strong belief in the ‘professor-model’, found it more difficult to accept EVS in the large-group lecture. This will be the subject of further investigation in the future. Students in the reflection stage (stage 2) demonstrated clear ideas about the char- acteristics of a good lecture. Features they identified include: Covers the material and covers it well. Information is well-explained. One where you are interacting. Depends upon lecturer style and charisma. Stage 3: ‘divergence’ This is the stage when students become more sophisticated about EVS, and is informed by the constructive reflections of the students who fell into types 2 and 3 in the previous stage. Both these types are very clear about both their reservations and enthusiasm, but type 3 students go further by presenting constructive ways of over- coming any perceived defects or redefining new roles and applications for EVS as an interactive tool. Those students in type 3 appear to be protecting the EVS ideology. In addition, the students who demonstrate divergence also showed evidence of high- level skill development by applying marketing concepts to extended use of EVS in lectures. They appeared to take marketing and customer orientation concepts and use them to: ● propose new uses for EVS to meet other needs; and ● suggest product improvement to the existing use of EVS. These students exhibited maturity in defining EVS in terms of their perceived learn- ing needs, and realised other educational needs which EVS could be used to service. EVS in lectures within business and marketing 15 The divergence stage symbolises a major shift in the way students think about EVS and their own learning. Most students, but particularly those who fell into type 3, want to be consulted, but indicate that they feel a part ownership of EVS with the lecturers who adopted this interactive lecture method. Those students in the divergence stage strongly defended the continued value of lectures as a teaching method, provided it included EVS. With me there are a lot of external influences. If there were no lectures I would not learn anything. I particularly like it [the EVS style lecture] when it is part of learning rather than part of assessment. I think PRS [Personal Response System] should be made more interactive. This final quote indicates how this student reflected on the experience of the EVS-style lecture. Students in stage 3 went beyond merely defending the survival of the lecture, especially the EVS-style lecture, to expressing the further potential of EVS and identifying other areas where this technology could be applied. Some suggestions that students in this group made for improving EVS lectures included: ● simplifying the start-up and close-down of a EVS lecture; ● equipping lecture halls with permanently fixed EVS equipment; ● using EVS to obtain individual responses to set questions in semester 1 lectures before moving to group interaction and group answers to set questions in semester 2 (or vice versa); and ● proposing different ways of allocating students to small groups with well-thought through reasons for doing this. In addition, students in stage 3 offered suggestions related to extending the use of EVS to other learning activities. Suggestions included: ● using EVS as a formal testing mechanism linked to student performance in the module; ● using it as a revision tool before examinations; ● applying EVS to modules that use problem sheets, such as in financial accounting and quantitative studies modules. We are exploring ways of using these suggestions in other learning activities. Finally, students in stage 3 also went beyond merely identifying the characteristics of a good lecture. They offered suggestions as to how feedback and interaction in the EVS-lecture could be expanded, including: ● enabling individual students to answer the set question and giving individual feed- back on answers; ● exploring ways of using whole group interaction; ● greater use of mini-cases—using them as a pre-test and then post-test; and ● integrating results into tutorial work. 16 G. Masikunas et al. Our data did not provide an explanation as to why students in stage 3 made these suggestions. From their responses, however, we can assume that the impact of EVS on their learning in interactive lectures has led them to conclude that its application in other learning activities would also be of benefit to them. Our study begins to shed some light on what provides meaningful ways of learning from the students’ perspective. The literature related to learning theory indicates ways of mapping our findings against the different explanations and paradigms about how learning best takes place and how different teaching methods represent different understandings of learning. From the findings related to the themes of team decision-making and interactivity, and the comments by a small minority of students that they would rather have worked on their own on the problems set during the lectures, we recognise that some skills may be best learned independently rather than as a part of group learning activities. There is little in the literature about this and, therefore, this is a subject for future study and exploration. However, given the studies by Mazur (1997) and Dufresne et al. (1996) and their emphasis on the importance of group learning, we would be reluctant at this time to lose the group learning element of the EVS lecture. Different paradigms of teaching It appears that EVS-style lectures combine a range of pedagogical approaches in an interactive learning experience. We therefore decided to try to map the students’ perceptions in our findings to a framework of different pedagogical approaches. Renshaw (1995) suggests a framework of four models for defining and describing learning and teaching. These are outlined in Figure 3. Figure 3. Contrasting approaches to learning and teaching (adapted from Renshaw, 1995) What this suggests is that different explanations of learning and teaching determine the place of a range of pedagogical approaches in a programme of study. Figure 3 also demonstrates differences between the traditional and the interactive EVS lecture. Traditional lectures sit firmly within Renshaw’s transmission model, while the interactive EVS lecture combines aspects of all four models. It is our contention that the EVS-style interactive lecture provides learning opportunities within all four models of learning and teaching, and therefore promotes deep active learning in the students. In addition, the themes that emerged from our findings show that students’ perceptions of EVS and its value span all four of Renshaw’s models of learning and teaching. For example, the comments from students within the theme consultation and participation indicate the value that students place on constructivist/social constructivist learning. The social (and academic) community in which the student lives and works plays a central role in helping students to understand new concepts in a meaningful way. In addition, Vygotsky (1978) emphasised the importance that other people in the student’s academic world have in developing the mental processes of that student. In addition, the theme from the student data, reinforcement and feedback, demon- strates the value students place on learning in a transmission model. The themes of EVS in lectures within business and marketing 17 The Transmission Model Learning is: Teaching is: Performing a task Giving information correctly Cumulative Sequential Receptive Direct From the outside in Structuring the environment Practising/performing Rewarding performance The Constructivist Model Learning is: Teaching is: Personal Setting challenging Understanding tasks Interpretative/ Observing/ selective interviewing Active Supporting learner activity Constructive Reviewing/integrating Helping learners to reconsider The Metacognitive Model Learning is: Teaching is: Mindful engagement Explaining meaning/experience The Social Constructivist Model (Socio-cultural Model) Learning is: Teaching is: Social Joint activity Assisted performance Scaffolding Creating dissonance Managing learning Modelling learning tasks strategically strategies Reflecting and monitoring reflection Supporting Adapting and Helping to apply transferring across contexts self-evaluating Providing evaluation criteria Participation Interactive and Guiding the Co-constructive conversation Self-regulating Assisting in group membership joint construction of meaning Evaluating with shared Demonstrating values community/ professional values Figure 3. Contrasting approaches to learning and teaching (adapted from Renshaw, 1995) 18 G. Masikunas et al. interactivity and team decision-making indicate the importance that students place on social constructivist/metacognitive approaches to learning and teaching, where learn- ing is influenced by social interaction and collaborative learning, and where cognitive skills are developed alongside content. The metacognitive model suggests that students need to develop and use strategies that enable them to monitor their own learning and understanding, and to consider alternative approaches to a task (Renshaw, 1995). Team decision-making in the EVS-style lecture requires students to justify to the rest of the group their individual choice of answer to the problem under discussion, and to consider alternative answers and justifications from other group members. Therefore, peer, team and collaborative activity are important metacognitive components of the learning process. Through the EVS-style interactive lectures we wanted to create a social- constructivist learning environment, where communication between all team members (students and teachers) encourages collaborative learning and peer student teaching, and where students can develop higher-order cognitive skills. We also wanted to minimise ‘back row syndrome’, where students deliberately sit at the very back of the lecture room to avoid the need to engage with the lecturer or the lecture content. In summary, our study seems to indicate that using EVS-style interactive lectures creates a deep active approach to learning within all four para- digms of learning and teaching. Conclusions EVS-style lectures contain features that promote active learning in lectures to large groups that were consonant with the beliefs of the two lecturers involved. These include the following: ● Learning ought to be based on learning theory that is applied to the large lecture situation. ● Students enjoy being active participants in an event that matches learning theory with teaching practice. ● Electronic voting is a very effective and efficient method for providing rapid feed- back that can be shared with students and their teachers. ● Questions posed provide feedback to learners and teachers about the effectiveness of lecturer’s teaching and its effect on student learning. ● As an aid to instructional technology, EVS is a relatively easy system to master, and provides a range of benefits that would be difficult to obtain by other methods. The interactivity in EVS lectures overcame many of the barriers to student learning associated with the large classes taught in the traditional lecture format. The EVS lecture opened up new dimensions for exploring effective pedagogy that provides a means of supporting interactive lectures to large groups of students and to ‘turn them on’ to learning. Our findings show that our initial aims and objectives for using EVS in lectures were met. EVS in lectures within business and marketing 19 Students, on the whole, perceive that EVS-style interactive lectures are an improve- ment on traditional lectures. The quantitative analysis (assessment results) also indicates that EVS-style interactive lectures can improve student learning and achievement. The identified themes, as well as the student responses to EVS, when mapped against different models of learning and teaching, indicate that EVS spans a range of pedagogical approaches. Furthermore, students are not a homogeneous group in their perceptions about, and responses to, EVS. This may raise issues about how the views of different types of students are managed. Finally, this particular innovation led us to review our understanding of learning, and to appreciate the value of other paradigms of learning and how they help to explain why students reacted so positively to a transformed learning environment. References Abrahamson, L. (1999) Teaching with classroom communication system—what it involves and why it works, invited paper presented at the 7th International Workshop ‘New Trends in Physics Teaching’, Puebla, Mexico, 27–30 May. Alvarez, M., Binkley, E., Bivens, J., Highers, P., Poole, C. & Walker, P. (1990) Case-based instruction and learning: an interdisciplinary project, in: Proceedings of 34th Annual Conference (College Reading Association), 2–18. Aronson, J. (1994) A pragmatic view of thematic analysis, The Qualitative Report, 2, 1. Available online at: http://www.nova.edu/ssss/QR/BackIssues/QR2-1/aronson.html (accessed 8 March 2005). Bligh, D. A. (2000) What’s the use of lectures? (2nd edn) (San Francisco, Jossey-Bass). Bloom, B. S. (Ed.) (1956) Taxonomy of education objectives. Handbook 1: cognitive domain (London, Longman Group). Bloom, B. S. (1984) The 2 sigma problem: the search for methods of group instruction as effective as one-to-one tutorial, Educational Researcher, 13(6), 4–16. Boyer, E. (1990) Scholarship reconsidered (NJ, Carnegie Foundation for the Advancement of Teaching). Boyle, J. T. & Nicol D. J. (2003) Using classroom communication systems to support interac- tion and discussion in large class settings, Association for Learning Technology Journal, 11(3), 43–57. Brearley, D. (1993) The case study: threat or opportunity?, Counsellor Education and Supervision, 33, 35–37. Brown, E. J. & Brailsford, T. (2004) Integration of learning style theory in an adaptive educational hypermedia (AEH) system, paper presented at ALT-C 2004, Exeter, 14–16 September. Butler, J. A. (1992) Use of teaching methods within the lecture format, Medical Teacher, 14(1), 43–49. Carney, C. (1995) Teaching with cases in the Interdisciplinary classroom: combining business language and culture, in: H. Klein (Ed.) World association for case method research and applica- tion (Needham, MA, WACRA), 117–127. Crouch, C. H. & Mazur E. (2001) Peer Instruction: ten years of experience and results, American Journal of Physics, 69(9), 2001, 970–977. Draper, S. W. & Brown, M. I. (2004) Increasing interactivity in lectures using an electronic voting system, Journal of Computer Assisted Learning, 20, 81–94. Dufresne, R. J., Gerace, W. J., Leonard, W. J., Mestre, J. P. & Wenk, L. (1996) Classtalk: a class- room communication system for active learning, Journal of Computing in Higher Education, 7, 3–47. 20 G. Masikunas et al. Ellliot, C. (2001) Using a personal response system in economics teaching, International Review of Economics Education, 1(1) 80–86. Available online at: http://www.economicsnetwork.ac.uk/ iree/1/elliot.htm (accessed 1 October 2005). Greenwald, B. (1991) Teaching technical material, in: C. Christensen, et al. (Eds) Education for judgment: the artistry of discussion leadership (Cambridge, MA, Harvard Business School), 193–214. Hake, R. R. (1996) Interactive engagement vs. traditional methods: a size-thousand student survey of mechanics test data for introductory physics courses, American Journal of Physics, 66, 64–74. Horowitz, H. (1988) IBM study proves use of student response systems increases attentiveness, paper presented at the Sixth Conference of Interactive Instruction Delivery for the Society of Applied Learning Technology (SALT), Orlando. Jarvis, P. (Ed.) (2002) The theory and practice of teaching (London, Kogan Page). Laurillard, D. (1993) Rethinking university teaching: a conversational framework for the effective use of learning technology (2nd edn) (London, Routledge Farmer). Liu, T. C., Wang, H. Y. & Chan, T. W. (2003) The features and potential of interactive response system, in: Proceedings of International Conference on Computers in Education (ICCE) (Hong Kong, IEEE Conference Publishing Services), 315–322. Marton, F. (1994). Phenomenography, in T. Husen & N. Postlethwaite (Eds) International encyclopedia of education (Oxford, Pergamon), 4424–4429. Mazur, E. (1997) Peer instruction: a user’s manual (NJ, Prentice-Hall). Nicol, D. J. & Boyle, J. T. (2003) Peer instruction and class-wide discussion: a comparison of two interaction methods in the wired classroom, Studies in Higher Education, 28(4), 477–473. Renshaw, P. (1995) Excellence in teaching and learning, in: B. Lingard & F. Rizva (Eds) External environment scan (Queensland, Department of Education). Sarah, M. D. (2002) Research into industry: four years of observations in classrooms using a network of handheld devices, in: Proceedings of IEEE International Workshop on Wireless and Mobile Technologies in Education (Los Alamitos, CA, IEEE Computer Society). Tellis, W. (1997) Introduction to case study, The Qualitative Report, 3, 2. Available online at: www.nova.edu/ssss/QR/QR3-2/tellis1.html (accessed 1 October 2005). Vygotsky, L. S. (1978) Mind and society: the development of higher psychological processes (Cambridge, MA, Harvard University Press). Yin, R. (1994) Case study research: design and methods (2nd edn) (Beverly Hills, CA, Sage).