Irish Journal of Technology Enhanced Learning Vol 3, Issue 2, 2018 1 ‘Lessons Learnt’: the student view in the #VLEIreland project Deirdre Ryan, Mary Immaculate College, University of Limerick Angelica Risquez, University of Limerick Abstract This paper offers an overview of general results and of longitudinal aspects of the data collected to date from student populations which generated from the #VLEIreland project. We argue how results shed light on common misconceptions around VLE usage, and what VLEs can offer from a pedagogical perspective. Building on previously published research results; this paper incorporates a total of 22 data collections, representing almost 24,000 responses, and deals with the general findings and some trends emerging from the data. Overall results year on year have shown that VLEs are used frequently and increasingly by the respondents, but much remains to be done in order to incentivize their use to generate student engagement, and move beyond the use of VLEs as content repositories. While being mindful of the methodological limitations of survey methods relying on self-selection, the breadth and scale of the data gathered across thirteen institutions over a decade offers a degree of reliability, and we attempt to mount a strong challenge to some common myths around the use of VLEs in our discussion. 1. Introduction The widespread investment in virtual learning environments (VLEs) by higher education institutions is often largely driven by political and contextual considerations, rather than by evidence of pedagogical impact. In order to inform the return on investment from VLEs from both an economic and learning perspective, it is crucial that patterns of actual user engagement by students and teachers are explored (Naveh, Tubin & Pliskin, 2012). We know from our experience as academic developers that VLEs and similar online tools can be harnessed very successfully to facilitate pedagogical approaches that emphasize student engagement and teaching innovation. But we have also learned from our practice that we need to be cautious when making claims about the use and impact of the VLE as a game changer. Early research into the use of VLEs (Blin & Munro, 2008; McGill & Hobbs, 2008; Sun, Tsai, Finger, Chen & Yeh, 2008) showed that VLEs are used, by and large, to facilitate ground-level pedagogical functions related to access and delivery of content, management of class administration and communications. Irish Journal of Technology Enhanced Learning Ireland, 2018. © 2018 Ryan, D. & Risquez, A. The Irish Journal of Technology Enhanced Learning Ireland is the journal of the Irish Learning Technology Association, an Irish-based professional and scholarly society and membership organisation. (CRO# 520231) http://www.ilta.ie/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license. D Ryan & A Risquez 2 Broadly, this can be attributed to lecturers’ tendency to make incremental changes to their practice when faced with new technology (Dutton, Cheong & Park, 2004; Jenkins, Browne & Walker, 2005; Kirkup & Kirkwood, 2005). Naveh et al. (2010) raise a critical voice and claim that “in fact, instructors can maintain their conservative teaching habits except for posting their course content on the website. From an organisational perspective, this can be done at low cost, yielding relatively high student satisfaction” (p. 132). However, much can be said about lecturers making incremental teaching improvements, as supposed to embracing disrupting change for change sake. Also, it is true that sometimes, it is the simple things that can make a big difference and enable student learning. This paper draws on the responses from the last coordinated iteration of the #VLEIreland Survey and compares these with previous trends in the data. The findings are discussed in terms of two common myths which center on attendance and the value of face-to-face teaching that are often cited by teaching and management staff about the use of VLEs. This paper provides evidence from the student perspective that may help to debunk these myths and add to the debate on the role of VLEs in the current teaching and learning landscape. 2. Methods The results presented here are based on the survey of student attitudes and usage of VLEs that has been run for a decade in the Irish higher education sector, which we came to term #VLEIreland. The survey instrument itself consisted of 20 questions, some of which had large numbers of sub-questions. The design incorporated a mix of yes/no and Likert question styles, and a mix of positive and negative questions to avoid common survey design bias. In addition to the quantitative data, there were a number of open ended questions that enabled the respondents to provide a little more depth to their input. Once coded, these responses provide a very useful qualitative addition and thus offer a greater degree of insight into the students’ perceptions and opinions. Surveys were conducted online, with the survey instruments generally disseminated via email to the whole student cohort. This paper draws on a pool of 23,879 responses across 12 institutions, collated from 22 survey instances from early 2008 to 2015 (administrations have not been jointly coordinated since). It is to be noted however that response rates varied from institutions to institution, from 18% to as low as 4%. The limitations of the survey design are noted in the editorial to this special issue in some detail. Nonetheless, we would argue that given the sheer breadth and scale of the data gathered across the large number of institutions, the data and resultant discussion has a degree of reliability, and certainly constitutes the largest dataset related to technology enhanced learning in the country. Results reported in this paper showcase the most recent set of cross-institutional results collected in 2015, with reference to differences found in earlier administrations of the survey. Their interpretation is enriched and informed by previous publications, the results and insights of which are also reported in this paper (Risquez et al., 2013a; Risquez et al., 2013b; Cosgrave et al., 2013). 3. Results/Findings Overall results year on year have shown that VLEs are used frequently and increasingly by the respondents: the proportion of students reporting that they accessed their system daily or a few times a week has increased from about 77% in 2008/09 to 94% in the 2015 administration. Irish Journal of Technology Enhanced Learning Vol 3, Issue 2, 2018 3 The utility of the VLE as a content distribution platform for curating and distributing teaching material has been evident year on year. As seen in Figure 1, a substantial proportion of students also report using the VLE for submitting assignments, which is probably associated with increasing adaptation of plagiarism detection systems like Turnitin or SafeAssign, which are often integrated with the VLE. However, the data suggests that VLE's have not yet become a widespread platform for more complex activities like online discussions or quizzes. The use of discussion forums has remained typically low, which may reflect the fact that moderating a discussion forum requires more input from the lecturer, and the resulting decrease in engagement might reflect increasing time pressures on teaching staff. Figure 1. If you use the VLE, why? What are the benefits to you? (2015) 3.1 Accessibility and Flexibility About three quarters of the students (74%) reported that the VLE helped to make their lecturers more accessible to them, with 28% reporting that they were more likely to communicate with their lecturer using the VLE. On occasions, the VLE provides a choice of communication for students unwilling or unable to communicate with lecturers in class. Obviously, increased class sizes bring more relevance for the need of such open, class based communication in larger classes, and it is a significant and overlooked benefit of the VLE as an ‘open office door’ for students to engage with their lecturers. About 45% of respondents felt that the VLE changed the hours they could study. This was corroborated by a large proportion of students who reported accessing the VLE from home (86%) and outside of ‘normal’ working hours (44% accessed the systems after 6pm and before 9am). Part time, mature and distance learners reap the benefits of enhanced flexibility of study time and location, obviously. One of the major trend changes during the longitudinal administration of this survey has been related to the very substantial increase in the reported use of mobile devices, with the D Ryan & A Risquez 4 flexibility they afford for VLE use, and blended learning in general. A growing 43% of 2015 respondents reported using a mobile device to access the VLE, compared with only 3% in 2008/09. In the meantime, laptop use has decreased by 10% during the same period from around 39.6% in early administrations. The questions around the use of mobile devices have been reviewed over the course of the research project to reflect the changing nature of device usage (for a more detailed exploration of its implications in greater detail, see Raftery (2018) in this issue). Finally, it should also be noted that the proportion of students who either strongly agree or agree with the statement “I can get help and support with the VLE” has changed little along the life of the survey. 3.2 Usage Issues Poor, inconsistent or non-usage of the VLE by lecturers emerged as a key issue in the open ended questions. Of the VLE’s users, the vast majority identified improved lecturer usage as the most important improvement they would like to see. Students recommend some conventions around file naming; files provided in consistent and accessible formats; making resources available in a timely manner; and updating resources. Simply putting up notes and/or links to other material and leaving that on the VLE indefinitely is frustrating to students, as one student noted: Sometimes the lecturers put up too many notes … so it's harder to find the relevant material and to make sense of all the extra writing/diagrams etc. Students welcome straightforward, comprehensive and consistent basic usage, which obviates the need for all staff to master basic VLE management skills. This being said, we observe that good progress is being made as 63% of students agree or strongly agree that their lecturers make good use of the VLE in 2015 (Figure 2), compared to 42% in 2008/09, so in general, the data seems to indicate that we are moving into a late majority, away from early adopters. Figure 2. My lecturers make good use of the VLE (2015) Irish Journal of Technology Enhanced Learning Vol 3, Issue 2, 2018 5 When asked what deterred them from using the VLE, lack of use by lecturers (23.8%) was by large the most common barrier to use identified. Students clearly would like their lecturers to make more use of the VLE (in 2015, 66% agree or strongly agree with this statement) as seen in Figure 3. This could be seen as a maturity of use of the system: as lecturers overwhelmingly are using the VLE now, students are increasingly focusing on the quality of this usage. Figure 3. I would like my lecturers to make more use of the VLE (2015) The student voice states a clear disagreement with the association of VLE use with class disengagement: results from the full dataset indicate that 88% of students disagree or strongly disagree with the statement that ‘Getting notes on the VLE makes me less likely to go to lectures’. Figure 4. Getting notes on the VLE makes me less likely to go to lectures (2015) Indeed, the learning advantages of facilitating access to lecture notes prior to class time is reported in open ended questions throughout the dataset as being one of the most positive D Ryan & A Risquez 6 effects of using a VLE on student learning. In this vein, analysis of the whole dataset showed that 73% of respondents felt that using the VLE helped to clarify what was covered in class, 63% indicated that it made it easier for them to learn, and 41% expressed the opinion that it helped them to understand how well they were doing. If used properly, especially when using a well thought out ‘flipped classroom’ approach, the VLE can promote a more meaningful engagement during class time as this student expressed: ..it just makes the lecture easier to understand when you have it on paper, then in the lecture you can focus on what the lecturer is saying, rather than scribbling the notes down. 4. Discussion It has been long expected that VLEs would be a means to helping students engage with each other and with their lecturers. This expectation has been largely revised in the current landscape dominated by social media, but still, the VLE retains an important role to play as the institutional supported, password protected, class enrolment based solution for teaching and learning. Many pockets of good teaching practice that make the most of the VLE features to encourage student engagement exist, but these are substantially more demanding of lecturer time than uploading class handouts and resources. Using the discussion forums in a constructive way to foster engagement, or using quizzes as formative assessment tools require more complex planning and pose an additional workload on the teacher. Time and the lack thereof are examined in McAvinia, Ryan and Moloney (2018) later in this Special Issue. Student reports have supported that there is potential for making more and better use of the VLE to bridge the online communication gap between teachers and students, supporting class based interaction. These findings suggest a model of VLE usage as being a platform for the solution of existing problems. Lecturers have used it to solve the 'paper mountain' problem of disseminating material to students, and in tandem are using it to address the reverse problem of managing and grading submitted work, using the technology to solve conventional teaching management problems. While being mindful of the methodological limitations of survey methods relying on self- selection, and its potential impact on survey results, we would argue that given the sheer breadth and scale of the data gathered across the large number of institutions over seven years, that the data and resultant discussion has a degree of reliability and we can safely attempt to mount a strong challenge to two common myths around the use of VLEs. 4.1 Using a VLE will decrease my class attendance VLEs are commonly misunderstood as a mere content distribution solution that adds nothing to the actual learning process. The students’ perceptions we have collected show disagreement with this and suggest that, in general, the use of a VLE adds value over and above content dissemination. During our daily educational development practice, academics often attribute VLE use to the attendance crisis. The results reported by Farrelly, Raftery and Harding (2018) in this Special Issue regarding the barriers to VLE use reported by staff also go in this direction. Irish Journal of Technology Enhanced Learning Vol 3, Issue 2, 2018 7 Their concern is that students will not be motivated to show up for class if notes and course materials are available online. This assumes, of course, that the main take-away from class attendance consists of information and information only, which has been strongly contradicted by our data. We have also learnt that while the availability of notes on the VLE is not in itself a deterrent to attending class, it can be regarded by students as a means of compensating for non-attendance when this happens. Non-attendance may be due to a range of factors, including illness or injury, part-time or full-time working, childcare arrangements and conflicting deadlines for coursework. Indeed, increasingly diverse student populations, balancing life and study demands, hugely benefit from having access to educational materials and use these as a safety net. The false VLE-nonattendance attribution needs to be studied more broadly and to think of student motivation in real terms, as in the insight offered by this student: Sometimes lecturers don't use it to its full potential…some are reluctant to put lecture notes up in the flawed attempt to make people come to class but if people don't want to be there they just won't come either way 4.2 If everything happens online, the face to face teaching will be redundant From the previous discussion follows another widely held misconception that, perhaps surprisingly, persists to this date: some educators fear that the progressive adoption of online learning approaches threatens their own professional identity and their relevance within the learning process. This is understandable amidst increased workloads and class sizes, concerns for job stability, intellectual property and growing accountability. A more productive debate can however be raised around what constitutes productive use of student time while attending class. Online and face-to-face teaching are not mutually exclusive; on the contrary, there is huge potential to blend them creatively. The widespread availability and support of an institutional VLE can lower the access barrier for many teaching staff with a reasonable learning curve, and provide a quick win-win solution. Overall, the data year on year tells a consistent story: the satisfaction of student use of VLE is intrinsically associated to the quality and quantity of the educational design invested in the first instance. This seems to indicate that, despite VLE usage tending to be highly focused on the content dissemination function, its use can help learners as a reliable, definitive curated repository of course material. It can be seen as a ‘one stop shop’ students can refer to at any time to access material before or after class times, allowing the teacher, and not the content, to be a the center of the learning process. 5. Conclusion The results summarized in this paper support the general view of the literature that VLEs prime function remains being an online repository. However, the importance of facilitating access to educational resources should not be underestimated, as clearly supported by the views that students have widely expressed. We must not forget that many faculty do not feel at home with online tools and that providing a user-friendly, non-technical, flexible and effective solution for content curation enables them to surmount some access barriers. Given increased pressures on teaching staff to deliver on the teaching, research and administration fronts, there is a lot to be said about reaping the low hanging fruit. During our practice we have however found that, while faculty often approach the VLE to upload content, this often D Ryan & A Risquez 8 leads to more creative uses including plagiarism prevention, faculty and peer interaction, more efficient feedback on learning, using learning analytics to monitor student retention at the early stages, and so on. Despite this, quite often we may become over-obsessed with the failed promises from the literature that did not seem to materialise, and we need to conciliate with the reality that, instead, the VLE is facilitating very effectively other -also- necessary functions. Clearly, if an all-mighty policy maker (we don’t see many of those too often though) was to decide to switch off VLEs suddenly, the disruption to the normal functioning of teaching and learning at the grass-roots would be huge. In the many conversations that have taken place around this research project, we have often referred to the potential of VLEs to metaphorically serve as a ‘Trojan horse’ to many other pedagogical developments in expected and unexpected ways. Irish Journal of Technology Enhanced Learning Vol 3, Issue 2, 2018 9 References Blin, F. & Munro, M. (2008). Why hasn’t technology disrupted academics’ teaching practices? Understanding resistance to change through the lens of activity theory. Computers & Education, 50(2), 475-490. Cosgrave, R., Costello, E., Risquez, A., Raftery, D., Logan-Phelan, T., Harding, N., Palmer, M., McAvinia, C., & Farrelly, T. (2013). Evolution of the VLE in Irish Higher Education: The Student Perspective. Paper presented to the European Association of Distance Teaching Universities (EADTU) Conference 2013. October 23rd-24th 2013. Paris, France. Dutton, W. H., Cheong, P. H. & Park, A. (2004). An Ecology of Constraints on e-Learning in Higher Education: The Case of a Virtual Learning Environment. Prometheus, 22(2), 131-149. Farrelly, T., Raftery, D. & Harding, N. (2018). Exploring lecturer engagement with the VLE: findings from a multi-college staff survey. [Special Issue, McAvinia & Risquez]. Irish Journal of Technology Enhanced Learning, 3(2), 11-23. Jenkins, M., Browne, T. & Walker, R. (2005) VLE Surveys: A longitudinal perspective between March 2001, March 2003 and March 2005 for higher education in the United Kingdom., 30, Oxford, England: Universities and Colleges Information Systems Association (UK). Kirkup, G. & Kirkwood, A. (2005). Information and communications technologies (ICT) in higher education teaching—a tale of gradualism rather than revolution. Learning, Media and Technology, 30(2), 185-199. McAvinia, C., Ryan, D. & Moloney, D. (2018). "I don’t have the time!" Analysing talk of time in lecturers’ use of the VLE. [Special Issue, McAvinia & Risquez]. Irish Journal of Technology Enhanced Learning, 3(2), 35-46. McGill, T. J. & Hobbs, V. J. (2008). How students and instructors using a virtual learning environment perceive the fit between technology and task. Journal of Computer Assisted Learning, 24(3), 191-202. Naveh, G., Tubin, D. & Pliskin, N. (2010). Student LMS use and satisfaction in academic institutions: The organizational perspective. The Internet and Higher Education, 13(3), 127- 133. Naveh, G., Tubin, D. & Pliskin, N. (2012). Student satisfaction with learning management systems: a lens of critical success factors. Technology, Pedagogy and Education, 21(3), 337- 350. Raftery, D. (2018): Ubiquitous mobile use: student perspectives on using the VLE on their phone. [Special Issue, McAvinia and Risquez]. Irish Journal of Technology Enhanced Learning, 3(2), 47-57 Risquez, A., McAvinia, C., Raftery, D., O'Riordan, F., Harding, N., Cosgrave, R., Logan- Phelan, T. & Farrelly, T. (2013a). An investigation of students' experiences of using virtual learning environments: implications for adacemic professional development. In C. O'Farrell D Ryan & A Risquez 10 & A. Farrell (Eds.). Emerging Issues in Higher Education III: From Capacity Building to Sustainability, Dublin: Educational Developers in Ireland Network, 99-113. Risquez, A., McAvinia, C., Raftery, D., Costello, E., O’Riordan, F., Palmer, M., Harding, N., Cosgrave, R., Logan-Phelan, T., & Farrelly, T. (2013b). Engaging students through the VLE: findings from the VLE Usage Survey 2008-2013. 6th Annual Learning Innovation Conference: Sustainable Models of Student Engagement – Rhetoric or Achievable? Aisling Hotel Dublin, 17 October 2013. Sun, P.-C., Tsai, R. J., Finger, G., Chen, Y.-Y. & Yeh, D. (2008). What drives a successful e- Learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers & Education, 50(4), 1183-1202.