A little and often: unanticipated outcomes from an ePortfolio evaluation impacting on early identification of risk and non submission of work Journal of Learning Development in Higher Education ISSN: 1759-667X Issue 2: February 2010 A little and often: unanticipated outcomes from an ePortfolio evaluation impacting on early identification of risk and non submission of work Megan Lawton University of Wolverhampton, UK Emma Purnell University of Wolverhampton, UK Abstract Since 2005 all staff and students within the University of Wolverhampton have had access to an ePortfolio system – PebblePad. In 2007 the University ran a HEA Pathfinder Project ‘Embedding ePortfolio at level 1’ which involved 1800 level 1 learners and 31 members of staff across all academic schools. The staff development activities used to develop teacher capabilities tried to mirror the student experience to great success. The knowledge gained from this project was then taken into a University-wide impact evaluation. The evaluation identified anticipated and unanticipated outcomes of pedagogic processes for personal development planning (PDP) and e-portfolio development. This paper will deal with two key areas: 1) the early identification of risk and 2) the reduction in non-submission of work. This work now feeds into the University’s Blended Learning Strategy (2008a). The overarching aims of this strategy are to enhance student learning and to improve each student’s learning experience. This is articulated via student entitlements of which the following are directly addressed by this work: All students should be entitled to: Formative assessment/s opportunities on line with meaningful electronic assessment feedback; Have the opportunity to participate in electronic Personal Development Planning (ePDP); Lawton and Purnell A little and often: unanticipated outcomes from an ePortfolio evaluation Journal of Learning Development in Higher Education, Issue 2: February 2010 2 The paper will look at how an ePortfolio system was used to develop learner skills, learner support issues and how staff from a central department worked with discipline-based staff to help support their students, finally addressing institutional strategy and support issues. Keywords: personal development planning (PDP); ePortfolio; progression; formative feedback. Introduction: the institutional context Within the University of Wolverhampton, personal development planning (PDP) was strategically placed within the Institution’s Learning and Teaching Strategies 2002-05, 2005-06 and 2006-10. In 2008 a Blended Learning Strategy was amended to the Learning and Teaching Strategy. The Blended Learning Strategy is articulated via student entitlements of which one is: students should have opportunities to engage with ePDP. The term ePDP was defined in the University of Wolverhampton Pathfinder Project (2008b) as PDP activities facilitated by electronic tools. As an outcome of the 2002-05 Learning and Teaching Strategy an institutional framework for PDP processes was established that set out that PDP should: Be a formative student-centred process; Provide a product that documented a student's achievement and experience at the university; Be integrated throughout the whole of the student experience at the university; Be developmental; Be used by students with tutor guidance. The framework was re-titled from PDP to PACE (Personal, Academic, Careers and Employability) planning and development (Figure 1) to reflect more closely the activities and outcomes, potentially at different levels, that the institution wished students to achieve. For example, at Level 1 the emphasis might be on personal and academic development and planning moving onto career and employability development and planning in levels two and three. In postgraduate studies, the entire PACE areas may be covered with an appropriate emphasis relating to the postgraduate experience. Lawton and Purnell A little and often: unanticipated outcomes from an ePortfolio evaluation Journal of Learning Development in Higher Education, Issue 2: February 2010 3 Figure 1. Formative student-centred process. At an institutional level it was decided that for both practical and pedagogical purposes that a paper-based system for recording PDP for all students would be inefficient and would go against the institutional blended learning approach to learning and teaching. As a consequence of this framework an electronic tool, PebblePad, was developed to support and deliver the PACE process. At the start of the academic year 2005/6 this tool was made available to all staff and students. In June 2007 a review of PDP at level 1 showed that all academic schools were using PebblePad in some way to deliver PDP processes in the level 1 curriculum. Between May 2007 and May 2008 the University ran a HEA Pathfinder Project to embed the use of e-PDP activities into the Level 1 curriculum of two modules from each of the 10 academic schools. This project included 31 members of staff and 1810 Level 1 students. Modules ranged from groups of 15 to the largest module with 350 students. The project was important to the institution as it helped us to move past ‘champions’ and early adopters of the ePortfolio system, building personal capability in a wider range of staff. See www.wlv.ac.uk/pathfinder for further details (University of Wolverhampton, 2008b). PACE Academic Skills for Learning, Academic literacies Careers Specific to the subject(s) Personal Metacognition, Learning preferences Employability Transferable skills Underpinned by Technology http://www.wlv.ac.uk/pathfinder Lawton and Purnell A little and often: unanticipated outcomes from an ePortfolio evaluation Journal of Learning Development in Higher Education, Issue 2: February 2010 4 What is an ePortfolio system? The concept of our ePortfolio systems is: A system which allows users, in any of their learning identities, to selectively record any abilities, events, plans or thoughts that are personally significant; it allows these records to be linked, augmented or evidenced by other data sources and allows the user to integrate institutional data with their personal data. It facilitates self-awareness, promotes reflection, and supports enrichment through commentary and feedback from the recipients of shared assets. It grows, develops and matures as the user accesses it, without constraint, over time. It provides tools for aggregating assets in multiple forms; for telling myriad stories to diverse audiences and ensures absolute user-control over what is shared, with whom, for what purpose and for how long. It is a personal repository; a personal journal; a feedback and collaboration system; and a digital theatre - where the audience is by invitation only (see ‘What is an ePortfolio?’ http://asp.wlv.ac.uk/Level6.asp?UserType=6&Level6=3302 on the university’s website for further details). Though the majority of ePortfolio and PDP activities in the taught curriculum take place within PebblePad, PDP processes are also found within the institution’s virtual learning environment WOLF (Wolverhampton On-line Learning Framework), in particular the use of group forums, and by the use of Word documents and email. There are key differences between WOLF and PebblePad that affect how the different pieces of software are perceived as supporting PDP processes. WOLF is described as providing a range of tools and facilities by which tutors can deliver enhanced learning material in support of classroom-based teaching and tutorial contact. WOLF does offer collaborative tools to facilitate discursive activity among classes and smaller workgroups. WOLF is tutor driven with staff being able to see all areas of the software including the ability to track and monitor student usage. http://asp.wlv.ac.uk/Level6.asp?UserType=6&Level6=3302 Lawton and Purnell A little and often: unanticipated outcomes from an ePortfolio evaluation Journal of Learning Development in Higher Education, Issue 2: February 2010 5 PebblePad is a personal learning space where no-one can access another person’s account or view materials unless shared by the owner on an individual basis or published to a Gateway (a Gateway can be thought of as an institutional space, the place where assets can be published to so that other users such as tutors, peers or external moderators can view, comment, collaborate or give grades as appropriate). Between September 2008 and Februrary 2009 an evaluation of pedagogic processes for PDP and e-portfolio development at the university was carried out. This evaluation took the PACE framework looking at the anticipated and unanticipated outcomes. The ePortfolio evaluation The evaluation drew on a variety of quantitative and qualitative data; some of this data was naturally arising, some was repurposed and some was especially gathered as indicated below. 1. Data sources: this included relevant internal and external documentation, internal reviews and external project reports, raw evaluation data gathered for other purposes and repurposed (including face-to-face, video and telephone interviews), mapping of practice, quantitative data from university systems (very limited) and discussions with key stakeholders including, students, staff and senior managers. 2. Formative evaluation: through creating a narrative that collated baseline data with staff qualitative data, examples of practice, key events and activities, the evaluation was able to analyse an initial, comprehensive, holistic overview of all the ePDP activity across the university that then led to the summative evaluation exercise. 3. Summative evaluation: this summative evaluation hones down the holistic big-picture narrative to address the key objectives in light of feedback and reflections from key stakeholders. Lawton and Purnell A little and often: unanticipated outcomes from an ePortfolio evaluation Journal of Learning Development in Higher Education, Issue 2: February 2010 6 The structure of the report took each element of the institution framework for the PDP processes and treated them as key objectives. Reference, where appropriate, was also made to the QAA policy statement (2000) to give sector-wide indicators of areas that an institution might want to address. Each objective was stated with associated outcomes. This was then followed by an examination of the evaluative feedback and any related issues raised. Recommendations are drawn from key stakeholders or from the implication from the evidence gathered. Where appropriate, images and examples of practice were included to illustrate or clarify an issue. Though the evaluation was about PDP/ePDP and ePortfolio processes and development, inevitably issues, language and functions related to the specific software PebblePad were mentioned. Where this happened, it was acknowledged and wherever possible transferable lessons to be learnt were identified. Two key findings of the evaluation that were not anticipated in the original framework were: The early identification of risk; The increased submission of work. Early identification of risk The majority of PDP activities take place in the taught curriculum though tutor-driven activities. At level 1 these are predominantly aimed at developing personal and professional attributes but also include the development of study skills, reflection, self analysis, confidence, self-esteem, sense of belonging, subject knowledge, collaboration, critical thinking, meta cognition and learning how to learn. Research and evaluation of a recent cross-university HEA funded Pathfinder project (2008b) looking at embedding ePortfolio in level 1, has shown that students like using electronic methods for their learning, teaching and personal development planning (PDP) but that they must see a value and benefit to undertaking those activities. Students found that by using an ePortfolio they had the ability to go backwards and look at their previous work, as well as plan forwards, both of which they saw as being really useful. 75% of the 606 students who Lawton and Purnell A little and often: unanticipated outcomes from an ePortfolio evaluation Journal of Learning Development in Higher Education, Issue 2: February 2010 7 responded to the Pathfinder evaluation questionnaire said they enjoyed participating in ePDP. The majority of PDP/ePDP activities have an element of both formative and summative assessment. Students commented that this affected their engagement with tasks. 75% said that they would not have undertaken PDP/ePDP tasks unless they were assessed. The most widely used teacher-led process for the production of documents that show a student’s achievement and experience at the university is by module-based scaffolded web folio templates (Figure 2). Scaffolded templates are used in all academic schools with both large and small class sizes. Cowan (1998) described what is attempted by the scaffolded template as ‘creating a constructive occasion for reflection-in-action’ and within that an opportunity for a structured dialogue for reflection. Characteristics of the scaffolded web folio templates are that they: Have multiple formative activities that are taken throughout the module; Create a whole web folio for summative assessment; Are written to guide students through a reflective process; Give examples of the language and writing style that the teacher expects this work to be produced in; Ask students to link evidence of their progress and achievements; Open a structured dialogue between staff and students and in some examples student to student dialogue. Lawton and Purnell A little and often: unanticipated outcomes from an ePortfolio evaluation Journal of Learning Development in Higher Education, Issue 2: February 2010 8 Figure 2. Page 1 of the scaffolded template for AD1007 research and study skills. The use of scaffolded templates at level 1 and in particular asking the students to write about themselves, their own abilities and aspirations, has allowed a dialogue between tutors and students to be opened up in language and writing styles that students can readily access without having to get to grips with academic conventions straight away. Another key element at level 1 is the recognition of the prior experience and learning that a student brings with them to the university. Students have commented as follows: Really enjoyable. Can write my feelings etc. down better than verbalising them! Using PebblePad helps to break tasks down for you into manageable chunks. Conventions for academic writing can then be introduced and explored as a ‘topic’ and as part of a repertoire of writing and learning skills that a student is expected to acquire while studying in HE. An academic literacies approach (Lea, 2004) has been adopted with the scaffolded templates acting as a support ‘tool’. The role of the CETL: Critical Interventions for Enhanced Learning (CIEL) central team has been to draw together a multi-disciplinary staff group that can develop models, offer examples and explore pedagogic practice that can then be transferred to a wider university audience. What we found meaningful from an academic literacies approach were the following: Lawton and Purnell A little and often: unanticipated outcomes from an ePortfolio evaluation Journal of Learning Development in Higher Education, Issue 2: February 2010 9 Understanding different settings, gaining a repertoire of practice, and how to apply this knowledge. All our courses are modular, which means that students have to be aware that they need to gain a repertoire of skills to produce work to different expectations from different subjects, modules or even lecturers. Moving away from a deficit study skills model towards reading and writing in a particular discipline. With an academic literacies approach we might reasonably be sure that students are unaware of academic writing conventions in a given subject in higher education, but that this does not mean that they can’t read and write. Understanding staff expectations. Members of staff may not always clearly articulate their expectations of academic writing in their discipline to their students. To do this staff needed to identify clearly what those expectations are and the reasons for them. Formative feedback. We needed to make sure that students are developing their academic writing skills. The use of formative feedback needs to be used to help them gain these skills. Active writing – little and often. We found that it was important to get students writing and then to give feedback to develop their repertoire of skills. Writing a little and often also reinforced the expectation that students in the modular scheme needed to work over the whole period of the semester. The elements of an academic literacies approach together with the use of an ePortfolio system to support PDP processes can be found within the level 1 taught curriculum via the use of scaffolded web folio templates. An unexpected outcome of this approach was the early identification of risk. Example 1: the School of Art and Design As part of the University’s Centre for Excellence in Teaching and Learning (CETL): Critical Interventions for Enhanced Learning (CIEL) (www.wlv.ac.uk/cetl), the Centre for Learner Development (CLD) was established within the School of Art and Design (SAD) based on their work that designed and developed an Individual Learning Profile (ILP) (Salter and http://www.wlv.ac.uk/cetl Lawton and Purnell A little and often: unanticipated outcomes from an ePortfolio evaluation Journal of Learning Development in Higher Education, Issue 2: February 2010 10 Peacock 2001, 2002; Salter et al., 2003). The ILP was originally a paper-based self- assessment document that asked a student to rate their own confidence in their skills for learning. The paper-based ILP also had a free-text box where a student was asked about their aspirations. This tool became part of the PebblePad software as a stand alone function or as something that could be used and linked to other applications such as action plans or web folios. In SAD they linked the ILP to a web folio and set it as an early task for students to complete and publish to their tutors. An outcome of embedding this task in a module as the first piece of formative self- assessment was an increase in students asking for support. In 2006, before the use of a scaffolded web folio template with formative activities, 44 students came forward for additional support. In 2007 (the introduction of the web folio) 198 students came forward rising in 2008, to 253 asking for help. Of the students being helped by the CLD in 2008 94% progressed successfully on their course. Of particular note is a culture shift in both staff and students that now sees asking for help not as a stigma or a sign of inadequacy but as a sensible thing to do to understand the expectations of studying in HE and as a way of improving your own learning. By using short formative activities, as well as the CLD, subject teaching staff and personal tutors were able to make contact with their students. This enabled the identification of support needs and any non-academic issues that might be affecting student learning. Issues highlighted included such things as IT facilities being stopped through non-payment of fees, non-attendance at the taught sessions, some misunderstanding of the instructions given and basic ITC skills which were hampering student engagement with the e-learning opportunities offered by the university. All issues raised were able to be addressed before it was too late for the student to complete their work. As Mary Lea (2004) comments, ‘supporting the relationship between writing and learning is not generally regarded as the remit of course designers. As subject specialists they are usually primarily concerned with course content and, therefore, often overlook the ways in which writing and textual practices more generally are central to the process of learning’. The use of the scaffolded template has knitted together the student activities, academic literacies and subject content and created a space for meaningful dialogue, and by that dialogue and early identification of risk with appropriate action being taken. For example, the School of Art and Design has a disproportionately large number of students with Lawton and Purnell A little and often: unanticipated outcomes from an ePortfolio evaluation Journal of Learning Development in Higher Education, Issue 2: February 2010 11 dyslexia. The module team has now developed podcasts (Dyson and Rhodes, 2008) to support the student activities in response to the needs of this group of students. This initiative has three main objectives: To improve tutor communication with students taught in large groups; To enhance the learning experience by recognising different student learning styles; To allow working students to ‘time shift’ their studies. Increased submission of work The use of scaffolded templates and Gateways has shown an increase in submission of work. A key element of this is the timely and appropriate use of student activities and formative feedback – feed-forward. Example 2: a large core level 1 module This is a school-wide core level 1 module with 440 students enrolled. It is taught in four iterations by subject and specialist study skills tutors: Week 1: students download and personalise a scaffolded ePortfolio template provided by their tutors. This included elements that a student must personalise and formative tasks that, when completed, will form the summative assessment. Week 3: Students were expected to personalise and use the ILP in their ePortfolio, submitting it to their tutor. Week 3: 210 ePortfolios were submitted. Week 4: students are reminded that ePortfolios needed to be submitted for formative feedback. Week 5: 330 students had submitted at this point. The 140 students who had not submitted were contacted by their tutor or a member of support department staff (electronically initially). Week 6: 110 students identified as potentially 'at risk', as after repeated reminders they still had not submitted any work. Week 12: All but 20 ePortfolios were received for assessment. Lawton and Purnell A little and often: unanticipated outcomes from an ePortfolio evaluation Journal of Learning Development in Higher Education, Issue 2: February 2010 12 This is just one illustration but many other examples of a similar nature can be found across the university. One of the key elements identified by staff and students is the use of timely and appropriate formative feedback. Formative feedback – feed-forward The CETL: CIEL activities in the School of Humanities, Language and European Studies have developed eMentoring and electronic feedback and feed-forward to students using an ePortfolio in PebblePad. Burke (2007) based student feedback activities around an on- line form which encouraged students to unpack the feedback, prepare for a tutorial, get more out of a tutorial, action plan to act on feedback and try out selected resources. As the feedback is attached to the student’s work there is no issue of the students failing to collect their work or feedback. The UK National Student Survey continues to record student dissatisfaction with feedback, both in terms of the quality of the feedback and the time lag between handing in and return on work (Yorke, and Longden, 2008) Students are encouraged to engage with the feedback and to edit and review work before summative assessment (Prowse et.al., 2007). In the feedback URL links to study skills sites (including to our own ‘Sharpen up your skills’ open web resource: www.wlv.ac.uk/skills) are given to students so that they have direct, concrete steps to help them improve their own learning. A key element is to get students to value the feedback that they receive, as Weaver (2006) points out this is not always the case. This can be exacerbated within a modular scheme where students may not see their assessment tasks as feeding into the whole course. Student comments on using an ePortfolio system to support the PDP process show that they can and do give a bigger picture perspective: Using PebblePad for PDP has helped me understand what I have learned in this module and in relation to other modules. The tasks were really helpful in helping me identify my strengths and weaknesses and where I need to focus on to improve these. I really enjoyed the ePDP tasks and intend to use the ePortfolio to help me structure my learning in the rest of my degree. http://www.wlv.ac.uk/skills Lawton and Purnell A little and often: unanticipated outcomes from an ePortfolio evaluation Journal of Learning Development in Higher Education, Issue 2: February 2010 13 Lessons learned and key considerations Research has shown that the CETL: CIEL central team has been most effective when working with subject staff within their own learning and teaching school, subject and module cultures. We have been able to see, and in a small way experience, some of the situations that make a difference to staff and students. This could range from the IT infrastructures and equipment available, student cohorts and profiles and subject specific issues. Our role has been one of capability and capacity building rather than of ‘doing’. We have tried to use an ethos of development mentoring (Megginson et al., 2006). The characteristics of developmental mentoring being: The mentor is more experienced than the mentee in relevant issues; There is mutual growth; The mentor helps the mentee do things for themselves; The mentor helps the mentee develop their own wisdom; The mentor helps the mentee towards personal insights to steer their own development; The primary outcome is one of personal development; Good questions are central; Any social exchange emphasises learning. In most instances we have not been able to operate in a full mentoring relationship with staff, mainly due to time constraints, although we have kept to its essence. Support for staff is crucial when you are expecting them to try new pedagogical and technical approaches in their teaching. Any risk is lessened if there is a stable IT infrastructure and adequate resourcing on a practical and logistical level. In addition, there must be a robust and systematic process for staff to receive technical help and guidance at the exact point of time when it is needed. The potential for student advisors to contribute as an extra layer of support is an area being explored currently by the Institute for Learning Enhancement (ILE), (the central learning and teaching department). Where student advisors are being used both staff and students have commented on the fact that they each have more appreciation of each others learning and teaching environments and experiences. Research and evaluation has shown that embedding PDP processes and the use of an ePortfolio system into the curriculum does not happen overnight. We were fortunate with Lawton and Purnell A little and often: unanticipated outcomes from an ePortfolio evaluation Journal of Learning Development in Higher Education, Issue 2: February 2010 14 the timing and funding of the Pathfinder project that enabled us to do a large scale implementation and embedding initiative, with 31 members of staff and over 1800 students. However, this was only possible as we had an established core of ‘ePortfolio experts/enthusiasts/champions’ who were able to act as mentors and thus enthuse and develop less ePortfolio experienced staff. Of particular use was the trialing and development of different pedagogic approaches that really used the ‘e-ness’ of the ePortfolio system for formative assessment, for example, to give quick, timely and meaningful feedback to students. This practice being very different from practices observed with paper-based portfolios which can often come to the tutor as a large completed entity. The ePortfolio evaluation and the Pathfinder project gave us the opportunity to work closely with a number of staff from across the entire discipline spectrum. Through this we, as a central department, have taken on board factors that enable and inhibit PDP processes and the use of an ePortfolio system within the taught curriculum, having a better understanding of subject specific issues. By the use of writing a little and often and getting feedback no matter how short this is, we have seen an increase in student submissions and an increase in the early identification of students potentially at risk due to a number of other social and academic issues arising while studying in HE. Partly this is to do with timely and appropriate critical interventions but it is also, we believe, because staff and student must see a value and benefit in undertaking what can be perceived as onerous tasks. References Burke, D. (2007) ‘Getting the most out of feedback’, pp. 36- 49, in Nutt, D. and Tidd, J. (eds.) 1st European First Year Experience Conference: April 2006. Teesside: University of Teesside. Cowan, J. (1998) On becoming an innovative university teacher. Buckingham: Open University Press. Lawton and Purnell A little and often: unanticipated outcomes from an ePortfolio evaluation Journal of Learning Development in Higher Education, Issue 2: February 2010 15 Dyson, J. and Rhodes, J. (2008) ‘Supported learning with podcasts – the key to the future’, CLTAD 4th International Conference Enhancing Curricula: Using Research and Enquiry to Inform Student Learning in the Disciplines. Lycée Francais, New York. USA 3-4 April 2008. Lea, M. (2004). ‘Academic literacies: A pedagogy for course design’. Studies in Higher Education, 29, pp 739–756. Megginson, D., Clutterbuck D., Garvey B., Stokes P. and Garret-Harris R. (2006) Mentoring in action: a practical guide for managers (2nd edn.). London: Kogan Page Ltd. Prowse, S., Duncan, N., Hughes, J. and Burke, D. (2007) ‘…do that and I’ll raise your grade’. Innovative module design and recursive feedback’, Teaching in Higher Education 12(4) pp 437-445. QAA (2000) Policy Statement of a Progress File for Higher Education [ONLINE]http://www.qaa.ac.uk/academicinfrastructure/progressFiles/archive/policysta tement 16/02/2009 Salter, P. and Peacock, D. (2001) An 'Individual Learning Profile' (ILP). ONLINE: http://wlv.openrepository.com/wlv/handle/2436/5933 (Accessed: 16 February 2009). Salter, P. and Peacock, D. (2002) Identifying and addressing the needs of art and design students at risk of underachievement in their incoming year of study. ONLINE: http://wlv.openrepository.com/wlv/handle/2436/3794 (Accessed: 16 February 2009). Salter, P., Peacock, D. and Ives, J. (2003) Identifying, monitoring and addressing the needs of art and design students at risk of underachievement in their incoming year of study. ONLINE: http://wlv.openrepository.com/wlv/handle/2436/5243 (Accessed: 16 February 2009). University of Wolverhampton (2006-2010) Learning and teaching strategy (2006 – 2010). ONLINE: http://www.wlv.ac.uk/PDF/celt-lt-stgy-2006-10.pdf (Accessed: 16 February 2009). http://www.qaa.ac.uk/academicinfrastructure/progressFiles/archive/policystatement http://www.qaa.ac.uk/academicinfrastructure/progressFiles/archive/policystatement http://wlv.openrepository.com/wlv/handle/2436/5933 http://wlv.openrepository.com/wlv/handle/2436/3794 http://wlv.openrepository.com/wlv/handle/2436/5243 http://www.wlv.ac.uk/PDF/celt-lt-stgy-2006-10.pdf Lawton and Purnell A little and often: unanticipated outcomes from an ePortfolio evaluation Journal of Learning Development in Higher Education, Issue 2: February 2010 16 University of Wolverhampton (2008a) Blended learning strategy. ONLINE: http://www.wlv.ac.uk/blu (Accessed: 16 February 2009). University of Wolverhampton (2008b) Pathfinder project: ‘embedding ePortfolio at level 1. ONLINE: http://wlv.ac.uk/pathfinder (Accessed: 16/02/2009). Weaver, M.R. (2006) ‘Do students value feedback? Student perceptions of tutors’ written response’, Assessment and Evaluation in Higher Education 31(3) pp 379-394. Yorke, M. and Longden, B. (2008) The first- year experience of higher education in the UK: final report. York: Higher Education Academy. Author details Megan Lawton is Assistant Director of CETL: Critical Interventions for Enhanced Learning. Megan has presented on staff development and pedagogy relating to the use of an ePortfolio both nationally and internationally and is currently the institutional lead for three international and national research projects relating to ePortfolio. Emma Purnell is a CETL: CIEL ePortfolio advisor. Emma has presented on ePortfolio pedagogy and the learner experience both nationally and internationally. She is currently a member of two international and two national research projects relating to ePortfolio. Her work is featured in the JISC ‘Effective practice with ePortfolios’ booklet and the JISC ePortfolio info kit. http://www.wlv.ac.uk/blu http://wlv.ac.uk/pathfinder A little and often: unanticipated outcomes from an ePortfolio evaluation impacting on early identification of risk and non submission of work Abstract Introduction: the institutional context What is an ePortfolio system? The ePortfolio evaluation Early identification of risk Example 1: the School of Art and Design Increased submission of work Example 2: a large core level 1 module Formative feedback – feed-forward Lessons learned and key considerations References Author details