Case Studies Compass: Journal of Learning and Teaching, Vol 5, No 9, 2014 Supervision Tracking: Improving the student and staff experience for projects and dissertations Dr Ray Stoneham and Aliyah Essop University of Greenwich Communication between students and their supervisors in an academic environment has evolved from face-to-face meetings in college rooms to multi-faceted interaction using a range of increasingly complex digital tools (e.g. email, messaging, video, social media, cloud storage). However, many of these tools contain data silos with access restricted to just one or a few users by passwords, which makes it difficult to share and integrate the data into a Managed Learning Environment (Stoneham, 2012). As a result, keeping track of this interaction is difficult for both staff and students, and is particularly difficult for academics who are supervising large numbers of students. This presents significant quality assurance issues, since concerns with supervision are often identified far too late to allow any effective action to correct the problem and investigation of complaints can be hampered by lack of access to data. In addition, the student experience is often far too dependent on the allocated supervisor. These concerns were identified over ten years ago in the School of Computing and Mathematical Sciences (CMS) and a Supervision Tracking system was implemented by one of the authors as part of the CMS Managed Learning Environment (Stoneham, 2013) to address the issues. The basic idea is that all interaction between students and their supervisor is recorded in the system, including uploads, feedback, details of meetings, messages and student blogs. Email is used only for notification of new entries needing attention. A dashboard for supervisors shows progress of their students and a dashboard for managers shows the progress of all students. Cases where little or no interaction is recorded can be easily identified and corrective action taken if necessary. The system has proved effective, with details of over 600 undergraduate and postgraduate projects being recorded each year. The transparency of the system, whereby all staff can view the progress of all students, has developed a sense of community among staff involved with project supervision. The system has been favourably mentioned by students in surveys and highly praised by external examiners and accreditation bodies, including the QAA and the British Computer Society. Evidence shows that there have been fewer complaints by students of poor supervision since the system was introduced. The authors of this article have used the system in different roles. As well as being the system designer and developer, Ray Stoneham has supervised over ten students each year using the system and, as programme manager, has had oversight of the progress of around 100 students each year on Masters projects. Aliyah Essop has used the system as a student on her Masters in Computing and Information Systems. Figure 1 is an example of the student profile page as viewed by a project manager, who can allocate supervisors and second markers. Other teaching staff see the same data, but without the editing options. Case Studies Compass: Journal of Learning and Teaching, Vol 5, No 9, 2014 Figure 1: Example student profile page Figure 2 is an example of part of a supervisor dashboard, showing the status of each of his/her students. It includes links to MP3 recordings of the student project pitch and viva, which students find to be particularly useful. Figure 2: Example supervisor dashboard Students view the same data (but only for themselves) and have appropriate functionality through a dashboard on the CMS Student intranet, as in Figure 3. Figure 3: Current student dashboard Case Studies Compass: Journal of Learning and Teaching, Vol 5, No 9, 2014 Although effective, the current system has a clunky user interface and has time-consuming maintenance overheads each year. The system was developed in a hurry and was not expected to last as long as it has. It was built using the appropriate technology of the time (classic ASP, Access, XHTML) and needs to be updated appropriately for use on mobile devices and to improve the user experience. Essop (2013) surveyed students and staff about the current system to determine how it could be improved. The research focused primarily on the functionality, effectiveness, usability, responsive design aspects and suggestions for future improvements in order to reflect good supervision practices. Responses were obtained from thirty-four users of the current system. The results of the survey indicate that, overall, stakeholders positively endorse and make good use of the current system.  75% of student respondents find that a dedicated area for projects motivates them to engage actively with their supervisors through an effective communication and feedback loop.  64% of respondents indicate that an improved user interface would facilitate and ease interaction.  71% of the respondents favour a responsive version of the current application, given the proliferation of diverse computing devices. Additional functionalities that would support the needs of users were also identified. These included processes to increase the efficiency of project supervision management and to enhance the student learning experience. Thus the features proposed for the new system mainly related to improvements of the user interface, added functionalities and deployment on other devices such as mobile phones, tablets and PCs. Figure 4 illustrates the proposed design for a supervisor dashboard. In this case, similar key indicators have been grouped into a tab which eases navigation and formats the information in a more coherent manner. Figure 4: Proposed student dashboard Case Studies Compass: Journal of Learning and Teaching, Vol 5, No 9, 2014 Essop interviewed staff in other schools of the University to ascertain whether they would benefit from a Supervision Tracking system for projects and dissertations. The staff interviewed included senior managers, directors of learning, programme leaders, project coordinators, supervisors and students, as well as the director of learning at a partner university. Based on the responses to these structured interviews, she developed a detailed specification for a Supervision Tracking system that could be introduced University-wide. A prototype system, with an improved interface, enhanced functionality and easy implementation on a standard web platform, was presented at the July 2013 APT Conference at Greenwich and received positive feedback. It is hoped that resources will permit the implementation of a more effective Supervision Tracking system in the near future. Discussions are currently taking place between schools in the University and the University’s central IT services, with a view to scaling up school-based systems to University-wide systems. Tracking of project supervision is one system being investigated. In addition, the merging of schools into a smaller number of faculties is giving further opportunity for spreading good practice in the use of intranet-based systems such as this. The only example of a similar system Essop (2013) found was developed in Ireland for off- campus supervision (MacKeogh, 2008), although Jaldemarka and Lindberg (2013) have subsequently published research into technology-mediated supervision of dissertations in Sweden. They showed that the supervision of students' undergraduate dissertation work has proved resistant to change, whether technology-mediated or not, but that students in general find such mediated participation helpful for supervision and they suggested that mediation by technology is a productive way to enhance the supervision of students' undergraduate dissertations. This is in agreement with our findings. Stoneham (2014) is currently developing an eTutorial system along similar lines to the Supervision Tracking system and it is being implemented for recording and monitoring the interaction of over 2,000 CMS students with their personal tutors. It has already demonstrated its benefits in helping the school to meet the requirements for tutorial support set by the University. The long-term strategy is to integrate the eTutorial system with the Supervision Tracking system and also with an existing online enquiry system, to develop a simple but effective Customer Relationship Management System to track as much communication between staff and students as possible. It is planned to integrate this further with a student learning analytics system (JISC, 2013; de Quincey and Stoneham, 2013) to give the ‘big picture’ of student engagement, with the aim of improving student retention and achievement. References de Quincey, E. and R. Stoneham (2013): ‘Learning Analytics and Dashboards: Identifying Potential Sources of Data and Metaphors’; Presentation at APT Conference, University of Greenwich, July 2013. Essop, A. (2013): ‘Design and Development of a Project Support System for the University of Greenwich’; MSc thesis, University of Greenwich. Jaldemarka, J, and J. Ola Lindberg (2013): ‘Technology-mediated Supervision of Undergraduate Students' Dissertations’, Studies in Higher Education, vol. 38, 1382-1392. Case Studies Compass: Journal of Learning and Teaching, Vol 5, No 9, 2014 JISC (2013): ‘How can I use analytics to benefit my students?’, JISC Inform, Issue 36, 13-17. Available at: http://www.jisc.ac.uk/inform/inform36/QandA.html (Accessed 14 February 2014). MacKeogh, K. (2008): ‘Using Moodle to Support Peer and Group Online Undergraduate Research Supervision’; EdTech 2008, the Ninth Irish Educational Technology Users Conference, 23 May 2008. Stoneham, R. (2012): ‘Managed Learning Environments in Universities: Are they Achievable?’ Compass (the Teaching and Learning Journal of the University of Greenwich), vol. 6, 45-54. Stoneham, R. (2014): ‘eTutoring’. Work in progression. http://www.jisc.ac.uk/inform/inform36/QandA.html