Irish Journal of Technology Enhanced Learning Vol 3, Issue 2, 2018 i Editorial: The #VLEIreland project Claire McAvinia, Dublin Institute of Technology Angelica Risquez, University of Limerick Introduction A decade after Martin Weller’s proclamation that ‘The VLE/LMS Is Dead’ (2007) was met with widespread acclaim and debate, virtual learning environments (VLEs) remain pervasive in most higher education institutions (HEIs) including those in Ireland. Indeed, some institutions here are currently in the midst of large-scale VLE projects, evaluating existing platforms and investing heavily in their new VLE choice1. Despite the arrival of numerous social media platforms and the smartphone era, use of these centrally supported platforms to facilitate technology-enhanced learning is very much alive. For ten years, a multi-institutional group of educational developers in Ireland has collaborated to gather student and teaching staff views on the use of VLEs across Irish higher education. In this Special Issue of the Irish Journal of Technology Enhanced Learning, we present findings and analysis of this work. The body of work we share with you contributes to the debate around the VLE’s state of health with a large amount of evidence and our experience at the coalface of technology-enhanced learning. We have sought an appropriate medium to communicate the body of work as a sequence, allowing freedom to present results but also to discuss analysis and identify key trends and themes. In the papers presented here, we aim to do exactly this by cross-referencing the different articles and taking full advantage of IJTEL’s open access format whereby readers can find out more about any of the related research. We are very grateful to the Editorial Committee of IJTEL for the opportunity to do this through a Special Issue. The papers presented here focus on the use of the VLE by students and teaching staff in Irish HEIs. For our purposes, VLE refers to the institutional web-based systems used to support teaching in most HEIs. These systems (sometimes called LMS or learning management systems) generally offer online access to course materials, discussion boards, and online assessment tools. In this research, a survey of students has been undertaken in 2008, 2009, 2011, 2012, 2013, and 2015 using a common set of questions. The institutions which have participated thus far represent a diversity of organizational histories and VLE systems, and results to date include the responses of more than 23,000 students. More recently, a staff version of the survey has been developed and piloted in some of the participating institutions. The 1 The past year has seen a swathe of high-level VLE projects around the island of Ireland. For example, UCC, DIT, CIT and Queen’s University Belfast have all launched large-scale consultations with staff and students to consider changing their VLEs. In at least one case a change has been decided, with migration to a new system underway. Irish Journal of Technology Enhanced Learning Ireland, 2018. © 2018 McAvinia, C. & Risquez, A. The Irish Journal of Technology Enhanced Learning Ireland is the journal of the Irish Learning Technology Association, an Irish-based professional and scholarly society and membership organisation (CRO# 520231) http://www.ilta.ie/. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license. C McAvinia & A Risquez ii resulting database constitutes the largest collection of information on student experience related to technology enhanced learning in Ireland to date. As a result, the Roadmap for Building Digital capacity in Irish Higher Education by the National Forum for the Enhancement of Teaching and Learning (2015) quoted this study, in order to support the claim that VLEs are not being used to their full pedagogical capacity. Previous confident predictions made about the end of the VLE also suggested it was not being used as fully as it could be. VLEs have been strongly critiqued as “shovelware” and as the dumping ground for notes and slides rather than the transformative online space to which educators had aspired at the end of the 1990s (Stiles, 2007). The newer VLEs and upgrades of the ‘traditional’ brands offer features such as integrated social media tools and e-portfolios, and have lost the visual cues tying them to the classroom, such as book and blackboard imagery. The regeneration of the VLE is remarkable, and we argue in this Special Issue that research into its adoption and use is therefore even more timely. It is essential to listen to those teaching and learning with these systems which are unique in their reach and in the extent to which they have become part of campus life. The results and arguments that will be presented through this Issue will provide a valuable platform to help deconstruct some of the claims and myths made about VLEs, deriving many positive conclusions from findings. In this editorial, we present the rationale, history of the VLE research project and its methods, which has been dealt with in more detail through previous dissemination efforts (see also Risquez et al., 2013a). We will also introduce briefly the papers to follow. Context: drivers for change and the evolution of the VLE The most significant policy statements published in Ireland in the past decade point to the expansion of higher education, while maintaining and improving the quality of learning and teaching, offering opportunities for engagement with communities and enterprise, and promoting increased equity of access as well as lifelong learning. These statements have pointed to the role of new technologies in realising our ambitions for higher education. Technology has certainly changed the ways and means by which all people can potentially be educated and fundamentally changed the sense of where people can be educated. The National Strategy for Higher Education to 2030 (DES, 2011) clearly articulated the role which technology should play in the provision of teaching and facilitating the learning experience. It describes a system which must be responsive to the needs of an increasingly diverse student population. The strategy acknowledges the advances which have occurred in Irish higher education in the past decade including developments in technology-supported learning which include the increasing use of VLEs to support learning (DES, 2011). There is a clear focus on the provision of flexible learning options for students including blended and online learning while acknowledging the requirement for the development of teaching skills and the provision of on-going opportunities to develop these skills. Ambitious targets have been set for the numbers engaging in higher education in Ireland between now and 2030 (DES, 2011). Enhanced use of technology as a means of assisting expansion of provision has also been called for (HEA, 2011, p.33). However, one should be mindful of accepting at face value the claims that the proponents of technology enhanced learning have sometimes made. As articulated in our submission to the group involved in the preparation of the National Strategy (National Forum, 2014, p.53), we must instead avail of opportunities to integrate research with teaching in order to enrich and enhance student learning. The desire to listen to the student perspective and use students’ voices to inform the continual improvement of their learning experience underpins the research presented in this Special Issue. Irish Journal of Technology Enhanced Learning Vol 3, Issue 2, 2018 iii It is important that, in addition to considering the local context for this research, we also consider the international research charting the adoption of VLEs elsewhere. Initial surveys in the early 2000s reflected the adoption of the VLE across further and higher education in the UK (Browne & Jenkins 2003; Jenkins, Browne & Armitage 2001; Jenkins, Browne & Walker, 2005). By 2010, VLEs or LMS were in use in 90-100% of higher education institutions in the US and UK (Brown, 2010; Williams van Rooij, 2011). However, by the end of the 2000s researchers who analysed critically the use of the VLE were beginning to suggest that its capacity to support innovative and active teaching and learning methods was extremely limited. Blin and Munro (2008) questioned why the VLE had failed to disrupt traditional lecturing practices. Conole, de Laat, Dillon and Derby (2008) suggested that VLEs were supporting and replicating lectures and notes distribution, rather than encouraging educators towards group learning or project-based assessment of students. Kirkup and Kirkwood (2005) found that technologies were in general adopted (at least in the first instance) to align with existing practice, and that evolution rather than revolution could be expected in the adoption of the VLE. However, research into the adoption of the VLE had limitations and gaps. There has been a tendency towards case studies of the use of the VLE in a particular discipline or with a particular group. There have been few studies which focus on the use of the VLE over a period of time: we tended to see snapshots and census-like countable data, rather than studies which give us a picture over a number of years of how a particular technology has been used. In the late 2000s and early 2010s, researchers began to reflect on what their own expectations of technology might have been, and whether these correspond with those of the users and practitioners in their universities. For example, Guri-Rosenblit (2005) has questioned whether we were measuring the impact of technologies in a reasonable way, when we tended to compare their use in campus-based institutions with that in distance institutions. She has argued convincingly that we need to take a different perspective on how online learning fits into a campus-based institution, when there are face-to-face teaching events comprising a large part of the timetable. These questions motivated a group of educational developers and technologists in the Irish Higher Education sector to evaluate whether there were some forms of ‘conventional’ VLE use that might well be appropriate to students and lecturers in the third level sector in Ireland, and to continue this evaluation over a period of years. The next section describes the research project that originated as a result. #VLEIreland: a sustained and sustainable collaborative project The project that we came to call #VLEIreland (https://vleireland.wordpress.com/) was born out of a persistent common need for better information on student perceptions of VLEs. It developed organically in response to this need, with operating structures at flexible levels of formality. There was no official project leader, and different team members contributed in proportion to their interests and capabilities. The research received no direct grant funding, had no fixed costs, and the work was done at the fringes of each team member's own role. As such, we operated as a peculiarly sustainable group, relying only on the continued interest of participants to maintain it. In 2014, we received a small grant through network funding awarded to the Irish Learning Technology Association (ILTA) by the National Forum for the https://vleireland.wordpress.com/ C McAvinia & A Risquez iv Enhancement of Teaching and Learning. We acknowledge this support for our work here as the funding afforded us an opportunity to meet face-to-face and discuss potential avenues for dissemination of the large body of data that we had collected to that date. Our dissemination efforts now materialise in this Special Issue, in order to contribute to the evidence base available to HEIs in Ireland planning for the development of technology-enhanced learning over the next number of years. Next, we will present a short story about how the project originated and grew. In 2008, Robert Cosgrave, then a researcher and educational developer under the umbrella of NAIRTL2 in UCC, launched a call to the Irish HE sector to collaborate with others around a survey of student attitudes and usage of VLEs. After an open invitation to all Irish tertiary institutions through ILTA, five institutions collaborated in the design of the survey and committed to running it and pooling the data. Participating institutions, on condition of anonymity, pooled their results for comparison. Data protection issues were addressed by stripping all individual identifiers out before pooling the data and ensuring the students were aware of what the data was being used for. In subsequent years, additional institutions joined the group, which currently includes 12 institutions and operates under the auspices of ILTA. Ethical issues in research with students and staff have been considered at each phase of the project and permission sought from the relevant institutional committees. The survey data has provided each institution with useful information on how their uptake and usage patterns compared with other institutions, in a framework which prevented abuse of the findings for marketing or public benchmarking. Research results have been disseminated through the years in order to generate impact, always preserving these ethical guidelines (for example, presentations and publications were to be approved by all members). Outputs have included a number of papers and keynotes in the EdTech national conference organised by ILTA, and at other national and international conferences, as well as publications (Cosgrave et al. 2011; Cosgrave et al. 2012; Cosgrave et al. 2013a, 2013b; Farrelly et al., 2015; Farrelly, Raftery & Harding, 2016; Risquez, Raftery & Costello, 2015; Risquez et al. 2013a, 2013b). The research has survived in hibernation for long periods as partner institutions run their internal surveys and gather data using the common survey tool, but it has also moved forward very quickly at times using a variety of remote collaboration tools (Skype, Wikis, Google Docs) to develop and share insights. The project developed a high degree of implicit sustainability by accident, because of its built in leanness and redundancy, and its ongoing value to the participants. Much of the value of this research is internal to each organisation, as it provides participants with a sense of how their own work and issues fit into a wider context of their peers. We anticipate that the survey created by the #VLEIreland project, which is currently integrated in the research strategy of ILTA3, will remain in use indefinitely as long as VLEs remain a subject of interest. Overview of research The research reported in this Special Issue draws on data from a survey instrument distributed first to students, and later adapted for distribution to staff in the participating institutions. The student survey instrument has evolved with time, and it currently consists of 24 questions, some of which have large numbers of sub-questions. The design incorporated a mix of yes/no and 2 Network for Advancing the Integration of Research, Teaching and Learning (http://www.nairtl.ie/) 3 http://ilta.ie/activities/research-projects/ http://www.nairtl.ie/ http://ilta.ie/activities/research-projects/ Irish Journal of Technology Enhanced Learning Vol 3, Issue 2, 2018 v Likert question styles, often addressing similar issues in different ways in different questions to accurately triangulate the students’ perspectives, and a mix of positive and negative questions to avoid common survey design bias pitfalls. The staff survey instrument was similarly constructed, to support comparisons where appropriate between staff and student responses. It consists of 22 questions, incorporating yes/no and Likert question styles, addressing the use of the VLE and touching on the use of other learning technologies, training and support available to staff and any barriers to their use or developing further their use of technology in their teaching. Here again, efforts were made to construct the survey instrument to afford some triangulation of perspectives and responses. In both surveys, there were also a number of open-ended questions that enabled the respondents to provide more depth to their input. Once coded, these responses provide a very useful qualitative addition and thus offer a greater degree of insight into the students’ and staff members’ perceptions and opinions. This special issue draws on a total of 23879 student responses across 12 institutions, collated from 31 survey instances from early 2008 to date. When the survey has been applied synchronously more than one (as is the case for nine of the institutions), some of the discussion compares data from their previous administrations. Finally, it is important to mention some inherent limitations that must be considered when interpreting the results presented in this Special Issue. Like all datasets, this one comes with specific caveats and biases which must be noted. Surveys were conducted online, with the survey instruments generally disseminated via both email and announcements within the VLE system. Response rates varied between institutions, which necessarily creates biases. Students with high digital literacy are more likely to respond, but there were few predictors of response rate amongst staff. As this is a survey on VLE usage, non-users of the VLE are likely to be under-represented in the sample. The data has not been weighted in any way, so institutions with large numbers of respondents may be over represented. The limitations of survey data need to be kept in mind when interpreting the findings. Therefore, response bias is acknowledged as a key limitation; we only have information from a relatively small subset of students, who are probably more likely to be more technically capable. To transcend this limitation, we believe it is important to draw on system data captured by the VLE itself, for example, the proportion of registered students who log in regularly, the proportion of modules with activity and so on. The details of these metrics will differ from system to system, and there is a substantial body of technical work in developing common metrics which can be used to draw comparisons. This work strand could draw the project in the direction of learning analytics, and deepen our collective understanding of system data and how it can be used to guide our professional development programmes and improve the overall student learning experience. It must also be noted that the initial survey was designed as one element of a much larger piece of work in one of the partner institutions, which drew on system data, staff survey data, student focus groups, key informant interviews and random sample staff interviews. While it is not practical to conduct such a detailed project on such a grand and open-ended scale, there is scope to add additional methods to the student to add depth and robustness to the data set moving ahead. Papers presented in this IJTEL Special Issue This Special Issue contains eight papers presenting analysis of the data from the #VLEIreland student and staff surveys over the past ten years. Ryan and Risquez present lessons learned from the perspective of student use of the VLE as reflected in the full dataset of some 24,000 responses and addressing myths around the use of the VLE as well as its pedagogical potential. Farrelly, Raftery and Harding present analysis of the staff survey data, examining the tools and C McAvinia & A Risquez vi features of the VLE used by lecturers, the barriers to use and emergent issues and concerns such as intellectual property ownership. Raftery and Risquez present a further analysis of the student data addressing student engagement, but controlling for some of the diversity of institutions involved in this research overall by conducting in-depth analysis of similar institutions from within the dataset. Harding’s paper focuses on the “digital turn” and how educational developers might address some of the issues faced by lecturers as they adopt and use the VLE in their teaching. McAvinia, Ryan and Moloney take a subset of the staff data to examine the discussion of time, and the lack of time available to staff to work with the VLE and other new technologies. Raftery examines what this research tells us about the use of mobile learning in conjunction with the VLE, and how student interaction with educational technologies is evolving at present. Logan-Phelan then turns to the issue of Learning Analytics in education, and taking relevant data from this research, examines the challenges to effective use of analytics including the need for effective design of blended learning in the first instance. McAvinia and Risquez conclude with an examination of the position of the VLE in relation to open education as reflected in the relevant data from this research. Conclusion This Editorial has introduced and described the project on which the research presented in this IJTEL Special Issue is based. At the outset, we expected to learn if our VLEs were doing ‘better’ or ‘worse’ than those of our peers. Instead, we have learned from the wealth of data provided by our students that we are all facing very similar issues, against which technical and organisational factors are much less important than we thought. The staff perspective is also very enlightening, as many of the authors have an internal requirement for staff survey instruments to help guide the planning of professional development programmes. As with the student survey, the development and deployment of a common staff survey instrument has recently enabled a broader understanding of the issues. Review of our survey tool is a key piece of work moving forward. Our initial question set focused on the VLE itself, and was implicitly framed around the undergraduate, lecture-centric learning paradigm. Potential uses of VLEs in support of project based learning or practical learning are not captured well, and will need to be considered. Similarly, technology moves on, and in light of that, the question set will need to be reviewed, to strike a balance between the value of being topical and current, and the need for consistency over time in a longitudinal study. This being said, the instruments, and the project as a whole, has proven to be of immense utility in our daily practice. The claims made by proponents of e-learning can induce feelings ranging from incredulity to paroxysms of hope for the future. On the face of it technology-enhanced learning certainly appears to offer ways and means of widening educational participation making education lifelong and lifewide. With the support of education developers, we have found overwhelming support for the view that VLEs seem to offer a useful platform whereby content, teaching and learning can be married into one meaningful alternative learning environment; and not just an environment regarded as supplemental to traditional learning. However, e-learning platforms exist within a milieu that is reliant on issues such as connectivity, access, lecturer and student experience and abilities, all of which impact on the potential effectiveness of said platforms. Whatever policies, protocols, think tanks, committees, legislation, commissions or memorandums are drawn up, designed or implemented, the reality for most learners and lecturers on the ground is frequently very different from the aspirations of such instruments or bodies. It is quite clear that aspirations and mission statements need to be turned into concrete executive decisions and actions. Irish Journal of Technology Enhanced Learning Vol 3, Issue 2, 2018 vii We have the pleasure to introduce the work of all the team who have contributed to this project through this Special Issue. We welcome additional participants to join the project team, from Ireland or overseas. If your institution is interested in becoming involved, please contact any of the authors. C McAvinia & A Risquez viii References Blin, F. & Munro, M. (2008) 'Why hasn’t technology disrupted academics’ teaching practices? Understanding resistance to change through the lens of activity theory', Computers & Education, 50(2), 475-490. Brown, S. (2010). From VLEs to learning webs: the implications of Web 2.0 for learning and teaching. Interactive Learning Environments, 18(1), 1-10. Browne, T., & Jenkins, M. (2003). VLE Surveys: A longitudinal perspective between March 2001 and March 2003 for Higher Education in the United Kingdom. UCISA publication. Conole, G., de Laat, M., Dillon, T., & Darby, J. (2008). 'Disruptive technologies', 'pedagogical innovation': What's new? Findings from an in-depth study of students' use and perception of technology. Computers and Education, 50, 511-524. Cosgrave, R., Risquez, A., McAvinia, C., Raftery, D., Harding, N., Costello, E., Farrelly, T., O’Riordan, F., & Palmer, M. (2012). Usage and uptake of virtual learning environments: findings from a multi-institutional student usage survey. Research paper presentation at EdTech 2012, NUI Maynooth, 31st May-1st June 2012. Cosgrave, R.J., Risquez, A., Logan-Phelan, T., Farrelly, T., Costello, E., Palmer, M., McAvinia, C., Cooper, R., Harding, N., & Vaughan, N. (2011) Usage and uptake of Virtual Learning Environments and Technology Assisted Learning Tools: Findings from a multi institutional, multi year comparative study. AISHE-J 3(1) Special Issue: papers from EdTech 2009. Available online at http://ojs.aishe.org/index.php/aishe-j/issue/view/3. Cosgrave, R., Risquez, A., Raftery, D., Logan-Phelan, T., Costello, E., Harding, N., Palmer, M., McAvinia, C., & Farrelly, T. (2013) The VLE Usage Survey Five Years In: Overview of Findings and Future Directions. Research paper presentation at EdTech 2013 Opening Up Education, University College Cork, 30th-31st May 2013. Cosgrave, R., Risquez, A., Raftery, D., Logan-Phelan, T., Costello, E., Harding, N., Palmer, M., McAvinia, C., Farrelly (2013) 'The evolution of the VLE in Irish Higher Education: The Student Perspective', in European Association of Distance Teaching Universities (EADTU) Paris, France, October 23rd-24th 2013. Farrelly, T., Raftery, D. & Harding, N. (2016) VLEs ‘What Lecturers Want and Do’, in EdTech 2016, Law Society of Ireland, May 26th-27th 2016. Farrelly, T., Raftery, D., Risquez, A., McAvinia, C., Harding, N., Costello, E., O’Grady, M., Ryan, D., McSweeney, D., & Logan-Phelan, T. (2015) Exploring lecturer engagement with the VLE: initial findings from a multi-college survey. Research paper presentation at EdTech 2015 Beyond the Horizon: Policy, Practice and Possibilities, University of Limerick, 28th-29th May 2015. Guri-Rosenblit, S. (2005). 'Distance education' and 'e-learning': Not the same thing. Higher Education, 49, 467-493. HEA (2011). Aligning Participation, Quality and Funding in Irish Higher Education. Dublin: Higher Education Authority. DES (2011). National Strategy for Higher Education to 2030. Dublin: Department of Education and Skills. Irish Journal of Technology Enhanced Learning Vol 3, Issue 2, 2018 ix Jenkins, M., Browne, T. & Armitage, S. (2001) Management and implementation of Virtual Learning Environments: A UCISA funded survey, available: www.ucisa.ac.uk/groups/tlig/vle/VLEsurvey.pdf Jenkins, M., Browne, T. & Walker, R. (2005) VLE Surveys: A longitudinal perspective between March 2001, March 2003 and March 2005 for higher education in the United Kingdom., 30, Oxford, England: Universities and Colleges Information Systems Association (UK). Kirkup, G. & Kirkwood, A. (2005). Information and communications technologies (ICT) in higher education teaching—a tale of gradualism rather than revolution. Learning, Media and Technology, 30(2), 185-199. National Forum (2014). Principles and First Insights from the Sectoral Consultation on Building Digital Capacity in Irish Higher Education. Dublin: National Forum for the Enhancement of Teaching and Learning. National Forum (2015). Teaching and Learning in Irish Higher Education: A Roadmap for Enhancement in a Digital World 2015-2017. Dublin: National Forum for the Enhancement of Teaching and Learning in Higher Education. Risquez, A., McAvinia, C., Raftery, D., Costello, E., O’Riordan, F., Palmer, M., Harding, N., Cosgrave, R., Logan-Phelan, T. & Farrelly, T. (2013a). 'Engaging students through the VLE: findings from the VLE Usage Survey 2008-2013', in 6th Annual Learning Innovation Network (LIN) Conference, Ashling Hotel Dublin, October 17th, 2013. Risquez, A., McAvinia, C., Raftery, D., O’Riordan, F., Harding, N., Cosgrave, R., Logan- Phelan, T., & Farrelly, T. (2013) An Investigation of Students’ Experiences Using Virtual Learning Environments: Implications for academic professional development. In O’Farrell, C. and Farrell, A. (Eds.) Emerging Issues in Higher Education III: From capacity building to sustainability. p.99-112. Athlone: EDIN. E-book available to view at http://www.edin.ie/?page_id=112 Risquez, A., Raftery, D. & Costello, E. (2015). The open dataset on students’ perceptions of virtual learning environments in Ireland: Collaborating to listen to the student voice. British Journal of Educational Technology, 46(5), 1070-1074. Stiles, M. (2007). Death of the VLE?: a challenge to a new orthodoxy. The Journal for the Serials Community, 20(1), 31-36. Weller, M. (2007, November): The VLE/LMS is Dead. Retrieved from http://nogoodreason.typepad.co.uk/no_good_reason/2007/11/the-vlelms-is-d.html Williams van Rooij, S. (2011). Higher education sub-cultures and open source adoption. Computers and Education, 57, 1171-1183. http://www.ucisa.ac.uk/groups/tlig/vle/VLEsurvey.pdf http://www.ucisa.ac.uk/groups/tlig/vle/VLEsurvey.pdf