Irish Journal of Technology Enhanced Learning Vol 4, Issue 1, 2019 Flipped learning in a civil engineering module: student and instructor experiences Bryan A. McCabe *a a Senior Lecturer, Civil Engineering, College of Engineering and Informatics, National University of Ireland, Galway. (Received March 2018; final version received August 2018) Abstract Flipped learning is a form of active learning in which the basic content and concepts are assimilated before scheduled classroom sessions, enabling more productive use of classroom time to cultivate problem-solving ability, a key requirement of engineering graduates. While the flipped learning approach has increased in popularity over the last five years, there are relatively few case studies for civil engineering modules in the literature, and none in the Irish educational context. The author’s experience (over the past six years) of delivering a third year soil mechanics module through flipped mode at NUI Galway is presented in this paper. The manner in which the flipped approach has been interpreted is described, including the development of short videos, the workshop format of the classroom sessions and the assessment methods used. Survey results have indicated that students found it to be a more convenient, engaging and effective learning experience, and surprisingly comfortable given the expectation of much greater activity in the classroom than would be typical of lectures. Finally, reflections from the instructor’s side, on both the benefits and the challenges, are conveyed. * Corresponding author. Email: bryan.mccabe@nuigalway.ie Irish Journal of Technology Enhanced Learning Ireland, 2019. © 2019 McCabe, B A. The Irish Journal of Technology Enhanced Learning Ireland is the journal of the Irish Learning Technology Association, an Irish- based professional and scholarly society and membership organisation. (CRO# 520231) http://www.ilta.ie/ . This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license. mailto:bryan.mccabe@nuigalway.ie http://www.ilta.ie/ http://creativecommons.org/licenses/by/4.0 B. A. McCabe 2 1. Introduction and literature review An accredited engineering degree is a professional qualification intended to produce graduates who can ‘do’ things, rather than merely ‘know’ things, from the outset of their careers. This ability to ‘do’ is best fostered through exposure to active learning within engineering modules/courses at university. Extensive research has shown that active learning strategies are associated with increased student engagement, critical thinking and improved attitudes towards learning (e.g. O’Dowd and Aguilar-Roca, 2009) and improved academic performance (e.g. Freeman et al., 2014). One emerging form of active learning is the ‘flipped’ or ‘inverted’ learning approach; according to Lage et al. (2000, p. 32),“inverting the classroom means that events that have traditionally taken place inside the classroom now take place outside the classroom and vice versa”. Applied to a university setting, students assimilate the basic module content and concepts outside the lecture room (through prescribed reading, or increasingly in a technology-enhanced format such as videos or podcasts), freeing up classroom time for activities aimed at developing their critical thinking and problem-solving skills. Consequently, the role of the lecturer is different in this model; the ‘sage on the stage’ (the instructor in didactic mode) is replaced by a ‘guide on the side’ (the instructor facilitates independent learning) (King, 1993). McWilliam (2008) advocates an alternative ‘meddler in the middle’ role, where the instructor supports self-managed learning but challenges students’ ideas to enrich the experience. Importantly, with increased contact time to circulate among students, instructors are better placed to respond more directly to specific learning needs (Bergmann & Sams, 2012). Most of the literature on flipped learning is very recent (since 2013), largely documenting modes of implementation used by academics and corresponding student perceptions. In separate studies, Bormann (2014) and O’Flaherty and Phillips (2015) reviewed over 30 and 28 peer-reviewed journal articles respectively on flipped learning, spanning a variety of subject areas (five studies were common to both). The key conclusions from these reviews were as follows: (i) The flipped learning approach was implemented in a variety of different ways, with no clear preference emerging for any one configuration or set of resources over another. However, the common thread was the emphasis on learning of a higher order (on Bloom’s taxonomy, for example) in the classroom than is possible in the traditional lecture format. Lower order activities were relegated to outside the classroom. (ii) Students perceived that flipped classroom activities were more engaging than lectures; this engagement arose from longer periods of higher-order cognitive activity. (iii) Prior experience of critical and independent thinking meant that students felt better prepared for the work environment and more confident in facing new challenges. (iv) There is little statistically significant evidence from quantitative studies that the flipped learning environment has an impact on student academic performance. In the context of (iv), Bormann (2014) advised the need for more comparative studies in which a common cohort of students is used, such as that of Willey and Gardner (2013), where the first half of a module was delivered in traditional lecture mode and the second half was delivered in flipped mode. As an alternative means of comparing the two approaches, Bormann (2014) suggested that two sections of the same module might be taught Irish Journal of Technology Enhanced Learning 3 concurrently in different modes. However, in this author’s opinion, there is no perfect comparative study; with the first method, the comparison is based on different content, some of which may call upon prior material learned through the other mode, while the second method is impractical in most research-intensive third-level institutions. In both scenarios, some ‘convergence’ of the two approaches is likely when delivered by the same instructor. Karabulut-Ilgu et al. (2017) synthesised findings from 62 publications on flipped learning specifically in engineering education. Of the 29 publications which compared the flipped and traditional learning approaches, 15 papers concluded that ‘flipped (learning) is more effective’, 4 papers concluded that ‘flipped (learning) is more effective and/or no difference’ and 8 papers concluded ‘no difference’. An analysis of variance was performed by Karabulut-Ilgu et al. (2017) based on average scores reported in 25 studies, which indicated that the average score for flipped instruction exceeded that for traditional instruction. The difference was not statistically significant when all studies were considered in a single data pool, but when they allowed for the fact that the authors were an inhomogeneous group, the difference was statistically significant (p = 0.042). Based on their review, these authors also summarized the benefits of the flipped model as flexibility, student engagement, increased interaction and development of professional skills; while staff workload, student resistance, technical issues and neglected material were among the particular challenges identified. Interestingly, only four of the 62 publications considered in the Karabulut-Ilgu et al. (2017) review relate explicitly to civil or environmental engineering modules, so further recounts of the application of the flipped learning approach in civil engineering modules are likely to be of great interest to instructors. In this paper, experience from six years of delivering a third year geomechanics module (at the Civil Engineering department, National University of Ireland, Galway) through flipped mode is presented; this is believed to be one of the first such deployments on an engineering programme at an Irish third level institution. The manner in which the flipped approach has been interpreted is described, including the development of short videos, the workshop format of the classroom sessions and the assessment methods used. While a comparison of examination results before and after the switch to flipped mode is inappropriate due to a number of background factors, general feedback from student surveys (tailored to the flipped learning approach) has been assimilated. Finally, the author’s experience of the process from the instructor’s side, both the benefits and the challenges, is considered in the paper. 2. Implementation of the flipped learning model 2.1 Introduction Geomechanics (or Soil Mechanics), the study of the engineering behaviour of earth materials, is a core topic on Civil Engineering degree programmes and is generally considered by students to be one of the more challenging modules. The introductory topics covered include (i) origin and classification of soils, (ii) phase relations, (iii) plasticity and grading, (iv) compaction, (v) effective stress, (vi) groundwater, seepage and permeability, (vii) consolidation, (viii) shear strength and (ix) lateral earth pressures. Effective stress is a particularly challenging concept which underpins topics (vi)-(ix) which relate to the design of foundations, retaining walls, slopes and embankments. B. A. McCabe 4 The flipped approach was first adopted for the introductory 3 ECTS CE312 Soil Mechanics module at NUI Galway in the 2012/13 academic year. The module content has remained substantially unchanged since 2012/13, despite the module name changes in Table 1. However, an institutional change to credit weightings arose in 2013/14, and student numbers and groups taking the module have changed over the past six years. Changes have been made to assessment formats and weightings partially in response (Table 1). Unfortunately, a comparison of examination marks before (up to 2011/12) and after flipping the classroom is considered inappropriate due to the following: (i) changes in participant groups as shown in Table 1 from 2012/13 onwards, and (ii) a gradual decline in CAO minimum and median points for the largest cohort (Civil Engineering students – 3BE) over a decade, in response to the declining appetite for the profession since the onset of an economic recession in Ireland. Upon initial contact with the students, the flipped model is explained and their engagement with the process is explicitly requested. In addition, a one-page briefing document is made available, which includes a link to a document by Educause Learning Initiative (2012) describing flipped learning. Feedback surveys indicated that 5% of students in the 2012/13 group had heard of flipped learning in advance of taking the CE312 module. Interestingly, only 17% of students in the 2016/17 group had heard of it, despite the substantial increasing publicity it has received (academic literature and online blogs) in the intervening period. Table 1: Details of six years of teaching Soil Mechanics/Geomechanics in flipped mode A c a d e m ic Y e a r M o d u le c o d e /n a m e E C T S 2 P a r ti c ip a n t g r o u p s3 E x a m w e ig h ti n g E x a m fo r m a t H o u r s p e r w e e k × n o . o f w e e k s 2012/13 CE312 Soil Mechanics 3 3BE (43), 3BSE (17), 3BCM (4), Total 64 80% Q1 + 2/3 3 × 8 2013/14 CE337 Introduction to Geomechanics 5 3BE (29), 3BSE (7), 3BCM (7), Total 43 60% 3/3 3 × 8 2014/15 CE3101 Geomechanics and Geology 1 5 3BE (17), 3BCM (9) Total 26 50% 3/3 3 × 8 2015/16 CE3101 Geomechanics and Geology 1 5 3BE (26), 3BCM (5) Total 31 60% 3/3 3 × 10 2016/17 CE3101 Geomechanics and Geology 1 5 3BE (32), 3BCM (5), 4BSE (14), Total 51 60% 3/3 3 × 10 2017/18 CE3101 Geomechanics and Geology 1 5 3BE (45), 3BCM (10), 4BSE (12), Total 67 60% 3/3 4 × 8 1 6 hours of geology delivered by another lecturer, not part of this flipped learning study. 2 ECTS refers to European Credit Transfer System; 60 ECTS per academic year. 3 BE = Civil Engineering, BCM = Project and Construction Management, BSE = Energy Systems Engineering (Civil stream); number (3 or 4) refers to the stage (year) of study. Irish Journal of Technology Enhanced Learning 5 2.2 Videos and pre-class activity A total of 15 videos were produced to cover both threshold concepts and basic content for the topics listed in Section 2.1 (the set of videos can be requested by contacting the author directly). The combined duration of 110 mins (average of 7.4 minutes per video) was arrived at organically; this short duration relative to the time spent lecturing previously (24 hours) was somewhat discomforting to the author before the first iteration of the module in flipped mode. However, it transpired that there was ample opportunity for significant development of the basic concepts through the workshop problems, so the total duration of the videos was not a concern after the first iteration. The videos were narrated MS PowerPoint files produced using Camtasia Studio software. Scripts were used in the preparation of all videos, most of which were produced to screencast.com. The remainder of the videos was produced to YouTube once a space limitation on screencast.com was exceeded. From 2013/14 onwards, the videos were also made available as .mp4 files through Blackboard Virtual Learning Environment (VLE). A review of over half a million videos by Fishman (2016) showed that engagement is independent of duration for videos up to 2 minutes long, while there is a decay in engagement between 2 and 6 minutes. However, most people who watch videos for 6 minutes are as likely to continue to watch up to 12 minutes, after which there is another drop off in engagement. 13 of the 15 videos produced had durations between 6 and 12 minutes. It is interesting to note that Google Analytics data for one of the Groundwater, Seepage and Permeability videos on YouTube (https://www.youtube.com/watch?v=lvIBR8wwyjI , 9:59 in duration, with 101,676 views as at 23/8/2018) suggested that the average view length was only 2:25 (24%). This viewing percentage is fairly typical of all six videos on YouTube. A series of online quizzes (through Blackboard VLE), comprising 3-4 questions per video, was introduced in 2013/14, to enable students to explore if they had comprehended the new concepts/knowledge in preparation for the classroom problems. These quizzes contributed to the overall mark. However, student feedback that year indicated that the workload expected compared to other 5 ECTS modules (continuous assessment comprising worked problems, quizzes and laboratories) was excessive, so these quizzes were left as optional/formative from 2014/15 onwards, in keeping with the approach of Willey and Gardner (2013). While this was believed to be a welcome development by all students, disappointingly, engagement with these quizzes since has been by and large limited to the high-achieving students. 2.3 Classroom workshops Engineering students tend to be visual, inductive (i.e. learn by exposure to examples) and active learners (Felder & Silverman, 1988). Therefore, the higher-order activity chosen for the classroom workshop sessions entailed discussion about and solution of geomechanics examples/problems, requiring the application of the concepts/basic knowledge assimilated in the videos. A workbook comprising 17 no. calculation-based problems is provided to the students at the outset of the module, two of which (related to seepage of water through soil) are shown in Figure 1. The set of problems was deliberately chosen with the breadth and depth of coverage of the module and some cross-referencing of topics in mind. In addition, they are used to provide a frame of reference for technical and practical discussions beyond the calculations, such as what soil compaction plant is used in earthworks projects, how sheet https://www.youtube.com/watch?v=lvIBR8wwyjI B. A. McCabe 6 pile or secant pile walls are constructed, or the role of geotechnical software in design. The students were provided with the parent PowerPoint files (with which the videos were produced) as a supplement, due to the impracticality of reverting to videos for diagrams or equations needed in the classroom environment. Figure 1: Sample workbook problems related to seepage in soils Any temptation to review the key points of the videos at the start of the workshop sessions was resisted by the author. However, students were encouraged to pose specific questions arising from their study of the videos either through the Discussion Board on Blackboard VLE or at the start of the classroom sessions, although they very rarely availed of these opportunities. This was not perceived to be a major problem, as any lack of understanding emerged and could be addressed during the workshop. The approach taken to solving each problem was student-led; students were invited to volunteer their ideas/rationale on how to tackle each stage of a problem to the class, sometimes after conferring with neighbouring students. Formal structured group work was difficult, owing to the presence of inflexible furniture in the classroom. The author helped tease out these ideas, as McWilliams’ (2008) “meddler in the middle”, often asking leading questions to tease out problems, questions from the perspective of others (client, contractor, consultant) or playing devil’s advocate to challenge initial erroneous perceptions. After some discussion, students then worked through the calculations in stages, while the author followed with partial solutions on a whiteboard. Unlike a tutorial, solutions only followed after significant student effort was confirmed (by walk-around or show-of-hands) and were deliberately left with some gaps, encouraging students to take responsibility for preparing a useful set of solutions themselves from which they could revise the material. Irish Journal of Technology Enhanced Learning 7 For variety, some additional videos were shown in classroom time to support the learning outcomes, such as one produced by the Norwegian Geotechnical Institute on the Rissa Landside, Norway, in 1978 (https://www.youtube.com/watch?v=3q-qfNlEP4A), another conceptualizing effective stress using a physical model by Prof. John Burland (https://www.youtube.com/watch?v=a-6YbkZJ5UY) and other videos showing sheet pile wall installation and piping failures. 2.4 Assessment In the 2012/13 academic year, the continuous assessment comprised five problems of a similar style and standard to those covered in the workbook, accounting for 20% of the overall mark (laboratory reports were included in a separate module in 2012/13). Since 2013/14, the module weighting for CE337/CE3101 increased to 5ECTS with the laboratory reports now incorporated. As mentioned in Section 2.2, summative quizzes were introduced for the 2013/14 academic year, but were used as formative assessment thereafter. 3. Student Feedback Feedback on the flipped learning approach was solicited at the end of each teaching period through a survey tool embedded in Blackboard. Findings from the 2012/13 (n=37 responses, 58%) and 2016/17 (n=36 responses, 71%) academic years are presented in the form of pie charts in Appendix 1. The trends are largely consistent for the two class groups. The feedback is considered under three headings: videos, workshops and overview. 3.1 Videos Typically around three-quarters of students accessed the videos through personal laptop and/or PC and over one-in-five though a university PC suite. The relatively limited use of smartphones or tablets was a surprising finding, given their greater portability. Notwithstanding the mode of access, 82% (2012/13) and 84% (2016/17) of students believed that the opportunity to watch the videos in their own time was either ‘convenient’ or ‘very convenient’. The ability to revisit videos was also appreciated: “It was reassuring knowing that if I had difficulty in part of a module, there was a video to explain it back” (CE337 2013/14). Replaying sections of videos may be responsible in part for the low average viewing durations referred to in Section 2.2. Nevertheless, flexibility is one of the most widely-cited benefits of the flipped learning approach (e.g. Velegol et al. 2015). 100% (2012/13) and 97% (2016/17) of respondents found that the quality of videos was ‘good’ or ‘very good’. The author would describe the videos as ‘fit for purpose’ rather than high quality productions; on the basis of the satisfaction levels with the videos, there is arguably little extra educational benefit to be gained from time spent to achieve the latter standard. This feedback has implications for the perceived ‘up-front’ investment in switching a module from traditional mode to flipped mode. A few students suggested that the videos would benefit from some simple worked examples. Videos were watched with a view to learning by 78% (2012/13) and 91% (2016/17); the majority relied on the videos alone but about one-quarter of these used other learning resources (e.g. books, internet) in parallel. On the other hand, 17% (2012/13) and 6% https://www.youtube.com/watch?v=3q-qfNlEP4A https://www.youtube.com/watch?v=a-6YbkZJ5UY B. A. McCabe 8 (2016/17) claimed to have watched the videos without any concerted effort to learn. In response to a query about video use by an external facilitator (Grouped Student Evaluation of Teaching, 2015/16), one class cohort suggested that some students reviewed the MS PowerPoint documents (from which the narrated versions were developed) only, to save time; this tendency has also been noted by Ossman and Warren (2014). The author also believes that the spacing of classes in the timetable is a factor in whether or not students review the videos in advance of class; double classes and classes on successive days limit the amount of preparation time available to students. 3.2 Workshops While some students respond openly (in front of the class) to questions posed in the workshops, others are either too reticent or insufficiently prepared for class to do so. Nevertheless, it is evident that a large proportion of those students who do not speak openly still contemplate and complete the problems. Approximately half of the students felt that they engaged fully with the problems during the workshop sessions (47% in 2012/13; 52% in 2016/17), while 37% (2012/13) and 42% (2016/17) claimed to have engaged partially. The workshop sessions always helped students understand areas with which they had difficulty in 39% (2012/13) and 31% (2016/17) of cases, and they sometimes helped in 56% (2012/13) and 55% (2016/17) of cases: “Doing examples in class allows us to ask questions about concepts we don’t fully understand, whereas if this was the normal approach, we wouldn’t be able to ask questions as easily” (CE3101 2015/16 Grouped Student Evaluation of Teaching). It is suspected that those who engaged partially in the classroom sessions (37%/42%) is a subset of the latter group. Almost 90% of students recognised the value of the workshop sessions compared to traditional lectures; 53% of students (both years) felt that the classroom sessions were ‘much more valuable’ than traditional lectures, while 34% (2012/13) or 36% (2016/17) felt that they were ‘more valuable’. Some student quotations highlighting the benefits for understanding and problem-solving ability include: “The flipped approach in the classroom was great because working through examples and interacting with the material actually made me think about what we were doing. In contrast, I tend to drift in and out of lectures and find it difficult to concentrate on what’s being said” (CE312 2012/13). “I found doing examples in class greatly improved my understanding of the subject and I engaged more than I do in other classes” (CE312 2012/13). “The primary difference was the increased focus on problem-solving which was extremely helpful compared to the traditional learning approach, which tends to feel like ‘here’s the material, understanding it is your problem’ ” (CE3101 2017/18). 38% (2012/13) and 53% (2016/17) of students felt that they were ‘much more likely’ to attend the workshop sessions compared to traditional classroom sessions, while 41% (2012/13) and 28% (2016/17) were ‘more likely’ to attend. Students felt that they were missing out on more than they would in traditional lectures by not attending: “In my opinion, all lecturers should try this method of teaching. It engages students as we are working out problems and thinking about them in class instead of just listening to lectures. The attendance at lectures is the highest of all 3 rd year modules, as people know they will learn by attending” (CE3101 2016/17). Irish Journal of Technology Enhanced Learning 9 3.3 Overview The flipped approach has met with overwhelming approval from most students such that the author has never contemplated a return to lecturing for this module: “Overall, I found this method of teaching to be an improvement on traditional lecturing and would be happy to see more lecturers adopt this approach” (CE312 2012/13). As part of the survey, students were asked which they felt was the most valuable part of the flipped learning process: primarily the videos (2012/13: 8%; 2016/17: 6%), primarily the workshops (24%/39%), or both videos and workshops (66%/47%). It was encouraging to realize in 2012/13 that 66% of the class appreciated that both components work in tandem; the lower value of 47% in 2016/17 may reflect greater instructor experience in conducting the workshops (while the videos have remained unchanged in the meantime). Despite the expectation of greater activity from students in the flipped format compared to a traditional lecture, 90% (2012/13) and 78% (2016/17) found the workshop learning environment to be either ‘comfortable’ or ‘very comfortable’, which is pleasantly surprising given that none of the class groups had prior experience of this approach. A number of students remarked that the flipped approach was a natural way of learning: “I was a huge fan of the flipped learning style of Bryan’s course. The way the theory is learned in the videos and applied in class is a natural way of learning in my opinion” (CE3101 2016/17). 4. Lecturer reflections Over the course of his teaching career, the author has steadily increased the level of student activity in scheduled class time, and the flipped learning trial in 2012/13 was a natural progression in this regard. The switch to the flipped approach has been a very rewarding and enjoyable experience for the author, and the workshops have challenged him to think more about the subtleties of geomechanics than had he continued to lecture the material. Over the last six iterations of the module, the maximum class size was 67 students; larger class sizes would undoubtedly render it more challenging to manage the workshop sessions effectively, and perhaps a teaching assistant would be helpful or indeed necessary in this respect. The initial time investment to switch a module to flipped mode was expected to be very high, on the assumption that 8-10 hours of videos would be needed to replace 24 timetabled ‘lecture’ hours. However, with the combined video duration of only 110 mins, and with some previous ‘lecture examples’ developed into workshop problems, this investment transpired to be much less onerous than anticipated. The emphasis on problem solving rather than lecture delivery has led to considerably reduced preparation time after the first year; and reminders have been made of topics for discussion or opportunities to ‘meddle in the middle’ from previous years. Lage et al. (2000) also noted this reduced preparation time for flipped classroom sessions compared to lectures, once the process is established. Unlike school teachers, most academics have no formal training in teaching, and therefore their ability to inspire a class through lectures can be limited in many cases. In the flipped model, the classroom time plays more to the strength of academics, which is their technical expertise, allowing them to address technical difficulties rather than lecture ‘delivery’. There is also an excitement about not knowing where the session is going to go and the time and flexibility to explore trains of thought (whether correct or incorrect) proposed by students B. A. McCabe 10 prompted by the workshop problems. For example, one notable student-led discussion focused around whether the over-consolidation ratio of soils, which has a theoretical minimum value of unity, is ever actually unity in reality. On the other hand, there can be slow days where it is difficult to get students to volunteer ideas openly to the class, even with classes that are generally well-engaged; this can be challenging for the lecturer to keep the session moving and there is potential for silences and wasted time. Engagement of the students in the workshop sessions was sometimes sporadic when they worked as individuals, but improved when they worked in groups. The author believes that the greatest scope for increased student engagement in his classes is through greater emphasis on group work, facilitated by a flexible seating arrangement where groups can be changed from session to session, and even within sessions. This also allows the instructor to pose questions to groups rather than individuals, which may be more comfortable for many students. 5. Conclusions The experience of flipped learning reported in this paper is overwhelmingly positive and student feedback suggests that the style of teaching/learning is appreciated by students and offers a more convenient, engaging and effective learning experience. Flipped learning offers great potential for problem-based and calculation-based programmes such as science and engineering. The benefits of the approach can be maximized if the physical classroom environment allows for formal group work, which was not fully exploitable in this study. It is hoped that the experiences shared in this paper will encourage instructors in Irish third-level institutions to consider this approach to facilitating student learning. Irish Journal of Technology Enhanced Learning 11 References Bergmann, J. & Sams, A. (2012). Flip Your Classroom: Reach Every Student in Every Class Every Day. International Society for Technology in Education. Bormann, J. (2014) Affordances of flipped learning and its effects on student engagement and achievement. Graduate Research papers, 137. https://scholarworks.uni.edu/grp/137 Educause Learning Initiative (2012) Seven things you should know about… flipped classrooms https://library.educause.edu/~/media/files/library/2012/2eli7081-pdf.pdf accessed 23/2/2018. Felder, R.M. & Silverman, L.K. (1988) Learning and Teaching Styles in Engineering Education. Engineering Education, Vol. 78, No. 7, pp. 674-681. Fishman, E. (2016) How long should your next video be? Accessed at https://wista.com/blog/optimal-video-length on 23/2/2018. Freeman, S., Eddy, S.L., McDonough, M., Smith, M.K., Okoroafor, N., Jordt, H. & Wenderoth, M.P. (2014). Active learning increases student performance in science, engineering and mathematics. Proceedings of the National Academy of Sciences of the United States of America, Vol. 111, No. 23, pp. 8410-8415. Karabulut-Ilgu, A., Jaramillo Cherrez, N., & Jahren, C.T. (2017) A systematic review of research on the flipped learning method in engineering education. British Journal of Educational Technology, Vol. 49, No. 3, pp. 398-411. King, A. (1993). From sage on the stage to guide on the side. College Teaching, Vol. 41, No. 1, pp. 30-35. Lage M.J., Platt, G.J. & Treglia, M. (2000). Inverting the classroom: A gateway to creating an inclusive learning environment. The Journal of Economic Education, Vol. 31, No. 1, pp. 30–43. McWilliam, E.L. (2009) Teaching for creativity: from sage to guide to meddler. Asian Pacific Journal of Education, Vol. 29, No. 3, pp. 281-293. O’Dowd, D.K. & Aguilar-Roca, N.M. (2009) Garage Demos: Using physical models to illustrate dynamic aspects of microscopic biological processes. Cell Biology Education, Vol. 8, pp. 112-122. O’Flaherty, J. & Phillips, C. (2015) The use of flipped classrooms in higher education: a scoping review. The Internet and Higher Education, Vol. 25, pp. 85–95. Ossman, K.A. & Warren, G. (2014) Effect of flipping the classroom on student performance in first year engineering courses. Proceedings of the 121 st ASEE Annual Conference and Exposition, Indianapolis, IN. https://peer.asee.org/effect-of-flipping-the-classroom-on- student-performance-in-first-year-engineering-courses https://scholarworks.uni.edu/grp/137 https://library.educause.edu/~/media/files/library/2012/2eli7081-pdf.pdf https://wista.com/blog/optimal-video-length https://www.sciencedirect.com/science/article/pii/S109675161500038X https://www.sciencedirect.com/science/article/pii/S109675161500038X https://peer.asee.org/effect-of-flipping-the-classroom-on-student-performance-in-first-year-engineering-courses https://peer.asee.org/effect-of-flipping-the-classroom-on-student-performance-in-first-year-engineering-courses B. A. McCabe 12 Velegol, S.B., Zappe, S.E. & Mahoney, E. (2015) The evolution of a flipped classroom - evidence-based recommendations. Advances in Engineering Education, Vol. 4, pp 1-37. Willey, K. & Gardner, A. (2013) Flipping your classroom without flipping out. Proceedings of the 41 st SEFI Conference, Leuven, Belgium. Irish Journal of Technology Enhanced Learning 13 Appendix 1: Feedback Responses Academic years 2012/13 (left, n=37) and 2016/17 (right, n=36) Which of the following is your principal means of accessing the videos? The opportunity to study the videos in my own time is: Please rate the quality of the videos: Which of the following describes how you engaged with the videos before the corresponding workshop session? 19% 80% 0% 1% University PC suite Personal laptop/PC iPad of tablet device Smartphone 25% 69% 3% 3% University PC suite Personal laptop/PC iPad of tablet device Smartphone 46% 36% 15% 3% 0% Very convenient Convenient Neutral Inconvenient Very inconvenient 42% 42% 16% 0% 0% Very convenient Convenient Neutral Inconvenient Very inconvenient 47% 53% 0% 0% 0% Very good Good Neutral Poor Very poor 50% 47% 3% 0% 0% Very good Good Neutral Poor Very poor 5% 17% 78% I did not watch them I watched half- heartedly I watched with a view to learning 3% 6% 91% I did not watch them I watched half- heartedly I watched with a view to learning 2012-13 2016-17 B. A. McCabe 14 To what extent have you engaged with the problems during the classroom sessions? Do the workshop sessions help you understand the areas you had difficulty with from the videos? How do the classroom sessions compare to traditional lectures? How likely are you to attend workshop sessions compared to a traditional lecture? For me, the most valuable part of the flipped learning process was: 47% 37% 16% Engaged fully Engaged partially Did not engage 52% 42% 6% Engaged fully Engaged partially Did not engage 39% 56% 5% 0% 0% Yes, always Yes, sometimes Neutral No, rarely No, never 31% 55% 11% 3% 0% Yes, always Yes, sometimes Neutral No, rarely No, never 53% 34% 10% 0% 3% Much more valuable More valuable Neutral Less valuable Much less valuable 53% 36% 8% 3% 0% Much more valuable More valuable Neutral Less valuable Much less valuable 38% 41% 18% 3% 0% Much more likely More likely Neutral Less likely Much less likely 53% 28% 19% 0% 0% Much more likely More likely Neutral Less likely Much less likely 8% 24% 66% 2% Primarily the videos Primarily the workshops Both videos and workshops No different to traditional format 6% 39% 47% 8% Primarily the videos Primarily the workshops Both videos and workshops No different to traditional format Irish Journal of Technology Enhanced Learning 15 How do you find the learning environment in the workshop sessions? 16% 74% 8% 0% 2% Very comfortable Comfortable Neutral Uncomfortable Very uncomfortable 20% 58% 14% 8% 0% Very comfortable Comfortable Neutral Uncomfortable Very uncomfortable