615 Assessing Law clinic – the use of digital patch text assessment as an alternative to traditional portfolios Karen Clubb 616 International Journal of Clinical Legal Education Issue 20 Introduction The paper presents the use of ‘patch text assessment’ as an innovative assessment method being ‘a series of integrated patch text assessments which are ‘stitched’ together with a final ‘patch’ to form a complete coherent piece of work. Each patch adds to the overall assessment.’1 The use of patch text assessment is presented within the context of a year long research project which aimed to assess the use of digital media to enhance the use of patch text assessment within a law clinic module in UK Law School. This paper presents a brief outline of the context of clinical legal education in relation to its application within the Derby law clinic module, outlining the current influences within the field of higher education that guide the assessment process and context. The paper then briefly defines ‘traditional’ portfolios and the concept of patch text assessment before considering these each in turn more fully in relation to their advantages and disadvantages in terms of assessment strategy. The paper then progresses to outline the objectives of the research project and the methodology applied to achieve these. This is followed by an analysis of the project findings to conclude as to the effectiveness of digital media to enhance the use of patch text assessment and any benefits derived from this approach in comparison with ‘traditional’ portfolios for assessment. The Derby Clinic experience The value of Clinical Legal Education (CLE) is evident in the number of law schools that now seek to make provision for student engagement in law clinics, and the research published to date as to the benefits of CLE as a pedagogic approach. The type and form of CLE varies, some institutions offer this purely as a non-credit bearing learning opportunity. Other law schools, such as Derby integrate this into a credit bearing module as an assessed component of the undergraduate law curriculum. Derby has for many years worked in partnership with local community legal service providers following a ‘placement style’ clinic model. Students are required to commit to a minimum number of hours, working within these community services alongside qualified solicitors and advice workers. The experiential model of learning offers a common rationale to all clinic provision, whatever the form. It offers students an insight into the law in practice and the ethical and professional context in which this is situated through their own participation in the provision of legal services. CLE affords students the opportunity to develop specific legal skills in a more meaningful way, learning from the context in which the practice of the skill is situated, enhancing their overall student learning experience and their employability. Academic discourse and research have to date focused on the value of CLE as a pedagogic methodology and its capacity to allow students to ‘experience and observe’ from ‘exposure to law in practice’2 Some institutions have phased this experiential dimension of clinical legal education to align with the stages of undergraduate academic progression.3 Others have captured the benefits 1 Clubb K., ‘The use of Patch Text Assessment in the Derby Law Clinic’ 2011. Poster presentation University of Derby Learning Teaching and Assessment Conference July 2011 2 Stuckey, Roy, Teaching with Purpose: Defining and Achieving Desired Outcomes in Clinical Law Courses (2006-7) 13 Clinical L. Rev. 807-838 3 Hall J., Kerrigan K., ‘Clinic and the wider law curriculum’ Learning in Law 2010 Available at http://www.ukcle.ac.uk 617 of experiential learning within developmental models of practice to further the attainment of professional competencies4. There has however been less consideration devoted to the design of assessment of this experience and the challenges this presents for both staff and students. The benefits acquired from CLE enhance student development and require careful assessment to capture the depth of the student experience and their skills development, mapped to curriculum benchmarks.5 Assessment strategies within undergraduate programmes in the UK have to align with quality standards set by the UK Quality Assurance Agency in the form of subject descriptors. The European Commission have sponsored a similar initiative to identify generic subject specific competencies within the European Higher Education Area, to support the Bologna process of mutual recognition of higher education and professional qualifications within the European area. Learning outcomes need to be carefully crafted to capture the richness of all that CLE can offer, reflected in an assessment method designed to match this. Not all undergraduate degree programmes, for example in the UK, require assessment of professional skills or practice per se, so there is no mandatory competency based assessment required. This is the case for most undergraduate law degrees in the UK which tend to be academic in focus. The current higher education context in the UK does however, require a consideration of how the degree programmes address the employability agenda. Whilst this is not a pressing issue in all jurisdictions, a consideration of the employability context has merit, if only to ensure that the clinic experience offered continues to reflect professional practice, and in light of clarifying student career aspirations. In the UK the Higher Education Academy (HEA), an independent champion of standards of excellence in teaching and learning, provides subject guidance to assist in addressing the employability context in law recognising that ‘law is taught both as an academic subject and as a precursor to gaining a professional qualification’.6 This is common to many jurisdictions. It is also recognised that only ‘approximately 50 percent of law graduates go on to train, but not necessarily to qualify, as solicitors or barristers.’7 The use of CLE is therefore valuable as an effective teaching methodology given its potential to enhance academic attainment in a relevant professional context. CLE offers a ‘real world’ experience that is particularly relevant in addressing the employability agenda in providing a realistic insight into professional practice, enabling students to apply and develop their knowledge to seek solutions to ‘real’ legal problems, often in relation to areas of law not previously studied.8 In the UK the Quality Assurance Agency that sets the quality standards for undergraduate education in the UK ,in the form of subject benchmarks which indicate the minimum requirement in terms of substantive areas and levels of assessment and performance required for the award of a law degree. Higher education institutions offering undergraduate law degrees retain freedom 4 Brooks S., ‘Meeting the Professional Identity in Legal Through a Relationship Centred Experiential Curriculum’ Education Challenge 41 U. Balt. L. Rev. 395 2011-2012, 396 5 Law Subject Benchmarks QAA 2007 Accessed 2/05/2012 6 Rees C., Forbes P., Kubler B., ‘Student Employability Profiles: a guise For Higher Education Practitioners’ HEA, CHIE September 2006, 97,98 7 Rees C., Forbes P., Kubler B., ‘Student Employability Profiles: a guise For Higher Education Practitioners’ HEA, CHIE September 2006, 97,98 8 Rees C., Forbes P., Kubler B., ‘Student Employability Profiles: a guise For Higher Education Practitioners’ HEA, CHIE September 2006, 97,98 Assessing Law clinic – the use of digital patch text assessment as an alternative to traditional portfolios 618 International Journal of Clinical Legal Education Issue 20 to ‘integrate assessment of key skills into performance on particular modules’ which may be ‘demonstrated by extra-curricular activities’ or by engagement in law clinics.9 The law subject bench marks allow the flexible integration of CLE as a teaching methodology. In reality the model of CLE adopted is more likely to be constrained by the staff skills mix and capacity for student supervision as well as the student demand, affecting the higher education institution’s ability to resource this. The potential benefit of participation in a law clinic, regardless of the model adopted would seem to afford the opportunity to assess most if not all of the subject benchmarks as well generic and transferable skills. The skills selected for assessment by a law clinic module are likely to be dependent upon the model adopted for the CLE experience and the form of legal service within in which CLE is situated, alongside the expected student role within this and degree of supervision afforded. Learning outcomes will no doubt reflect the skill areas that the academic team assess as being capable of being developed by the engagement in the law clinic experience, matched by assessment methods which are capable of evidencing their attainment. There is no set formula for this and learning outcomes will be institutionally specific and sufficiently broad to capture evidence of attainment of as full a range of skills as possible. Assessment methods – defining key terms Law Clinic offers the potential to assess a range of skills and knowledge, some generic and some legal, through assessed portfolios. Portfolios are ‘simply a collection of self-selected student work. Its primary purpose is to provide avehicle for students to reflect’.10 Portfolios have been described as: ‘a purposeful collection of student work that exhibits the student’s efforts, progress and achievements. The collection must include student participation in selecting contents, the criteria for selection, the criteria for judging merit, and evidence of student self-reflection’11 The work students select for inclusion in the portfolio must be of sufficient quality to satisfy the relevant learning outcomes and include ‘those pieces that best reflect their learning.’12 Portfolios are ‘a useful way of getting students used to writing reflectively, introducing them to the idea of providing evidence for their reflection.’13 Portfolios have been described as having two dimensions, a positivist one where the ‘purpose of the portfolio is to assess learning outcomes and those outcomes are, generally, defined.’ Here the portfolio acts as ‘a receptacle for examples of student work which are used to infer what and how 9 QAA Law Subject Benchmarks 2007, 4.7 Available at http://www.qaa.ac.uk/Publications/InformationAndGuidance/Documents/Law07.pdf 10 Johansen S ‘What Were You Thinking?”: Using Annotated Portfolios to Improve Student Assessment’ (1998) 4 Legal Writing J. Legal Writing Inst. 123, 135 11 Pauleson F., Pauleson P. Meyer C ‘What makes a portfolio a portfolio?’ New Directions for Teaching and Learning , 2008(74) 79-87 12 Johansen S ‘What Were You Thinking?”: Using Annotated Portfolios to Improve Student Assessment’ (1998) 4 Legal Writing J. Legal Writing Inst. 123, 136 13 Hinett K ‘Developing Reflective Practice in Legal Education’ UKCLE 2002. 619 much learning has occurred.’14 Additionally the constructivist dimension assists in going beyond merely recording learning but assisting the learning process. Here portfolios capture and enhance ‘the learning environment in which the learner constructs meaning’ and facilitate the presentation of a record of the processes associated with learning itself. Reflective portfolios have the advantage of potentially capturing both these dimensions. A different approach – Innovative assessment and Patch Text Assessment Patch text assessment (PTA) has been used in a number of different contexts but is common in the context of the education and training of teachers and the medical professions. It was developed as a tool to facilitate the development of the reflective process to assist reflective writing as a tool to further professional development and competence in professional practice. The capacity for reflection on professional practice is highly relevant to the vocational training of these professions, which necessitates a critical reflection on the professional role within the context of service delivery. PTA has most commonly involved learners sharing and commenting upon each other’s work, incorporating both peer review and tutor review and feedback within the PTA process. These rich sources of feedback offer the potential for students to improve their capacity for critical and in depth reflection by promoting critical enquiry and the defence of their reflections amongst their peers. The concept of PTA employs an assessment comprised of a series of single ‘patch texts assessments’, patches of reflective critique, that when finally ‘stitched’ together demonstrate the attainment of specific learning outcomes.15 The final work is not a collection as with the portfolio but a complete and coherent whole,16 one piece of work. The relationship between the different patches as a product of learning and an assessment method contributes to the structuring of the learning process. Ovens describes PTA as the Integration of written pieces across the module, which demand critical and personal engagement, and have been the subject of peer and formative feedback, to produce a structurally unified reflective synthesis.17 The final piece is a synthesis of learning derived from the previous patches. Winter has described PTA as representing an ‘attempt to combine the coherent structure of the essay with the open-ness of the portfolio.’18 The benefit of PTA lies in the perspective and understanding at the outset of the relationship of each individual assessment task and the skills that this draws upon and the relationship between these elements and the final piece of work and end learning outcomes. This requires the synthesis 14 Paulson, F.L. & Paulson, P. (1994) “Assessing Portfolios Using the Constructivist Paradigm” in Fogarty, R. (ed.) (1996) Student Portfolios. Palatine: IRI Skylight Training & Publishing, 36 15 Winter R. (2003) ‘Contextualising Patchwork Text: Addressing Problems of Coursework Assessment in Higher Education, Innovations in Education and Teaching International, 40(2)112-122 16 Brunsden V., Goatcher , J., 2008. Enquiry based learning . In: The Higher Education Academy Psychology Network New Lecturers Forum, Aston University, Birmingham, January 2008 .   17 Ovens P. (2003). Using the Patchwork Text to develop a critical understanding of science. Innovations in Education & Teaching International, 40(2), 133-143 18 Winter R 2006 A Patchwork Text online Periodical Service Company (www.periodicals.com) Assessing Law clinic – the use of digital patch text assessment as an alternative to traditional portfolios 620 International Journal of Clinical Legal Education Issue 20 and coherence between the learning outcomes and the PTA design at the outset. The advantage of PTA potentially over a portfolio is that the former focuses on the learning process, as well as the learning product/ outcome, providing a clear and purposeful structure to the assessment process. Indeed PTA readily requires students to make full use of formative feedback and the assessment process can drive the learning process. PTA is therefore relevant to CLE and an experiential law clinic module providing a structure to support the learning experience to enable students to extract the meaning from their reflection on their learning in context. Experiential learning whilst rewarding and motivating for students requires a greater degree of student autonomy and responsibility for their learning, through active participation rather than a passive consumption type approach. It potentially yields richer deeper learning by structuring the assessment process and using the process of reflection to both evidence and enhance learning, raising the student’s awareness of further learning opportunities. PTA potentially captures the transformative and developmental learning associated with the law clinic experiential learning and uses these to drive the learning experience and student performance. Traditional portfolios Portfolios19 are a common form of assessment in undergraduate education and the range of learning technologies allows for the use of e-portfolios. The traditional paper based portfolio is regarded as a collection of student work on their experiences in the law clinic experience over the period of the module. The expectation is that students reflect on the development of their skills and also their personal development during the clinic experience, reflecting on the impact of this on their learning, particularly in relation to the role of the lawyer, their future career and professional practice. Portfolios can readily be tailored to assess a diverse range of learning experiences, to evidence the attainment of a breadth of skills and the extent of learning according to the specific module learning outcomes, offering the possibility of multidimensional assessment tool.20 The reflective element of writing in the reflective diary or journal, is a critical element that draws together the student’s observations, perceptions and critique of their experience. This element of the portfolio demands more than demonstrating knowledge and new insights through a mere descriptive articulation of learning. It further requires a demonstration of critical analysis and synthesis of learning, drawing on the relevant features of the context of the experience and their impact on the student’s learning and their understanding of the context of legal practice. The reflective element of the portfolio is the mechanism by which the students evidence their understanding of theory and practice21 and its integration and relevance to the specific context of learning and the wider context of delivery of legal services and professional practice. Reflections are the ‘student voice’ demonstrating the student’s knowledge and the application of this within the given legal context. That which the tutor is unable to observe or know in relation to the student performance and knowledge acquired in the clinical legal practice/education setting, the student can convey in their reflections. 19 Portfolios are defined as “A collection of work that relates to a given topic or theme, which has been produced over a period of time” QAA Explaining contact hours Guidance for institutions providing public information about higher education in the UK, August 2011. 20 Venn, J. J. Assessing students with special needs ( 2002 2nd ed NJ: Merrill) 21 Santos M , ‘Portfolio Assessment: And the Role of Learner Reflection’ English Teaching Forum 35 (2): 10–14 621 The process of reflection, whilst sufficient to demonstrate the required learning outcomes, is by its nature, the student’s perspective of their learning and/ or performance. This requires further validation by reference to external evidence to support the student reflections. This is especially the case where students are expected to demonstrate application and development of specific legal skills. There is however the danger that the portfolio may become a ‘tale of two halves’ in relation to the student experience, comprised of two separate elements and processes. The separation of the evidence / record of practical skills development and learning from the parallel reflective process on learning can result in the portfolio becoming a collection of individual episodes or moments of learning rather than a seamless representation of the whole clinic learning experience as one ‘joined up’ piece of work. Consequently portfolios are not always perceived as leading to ‘integrative’ assessment of /or learning.22 In part this may arise from the quality of the reflective critique and/ or selection of episodes of learning for reflection and inclusion in the portfolio. The problem in relation to portfolios, is that students may focus on one time frame of the clinic learning experience, rather than to reflect more widely back and forward in time to link their learning from these experiences. The reflective process is critical to learning, to enable students to make sense of their clinic experience and to synthesise new knowledge, understanding and insights into their portfolio through their reflections which evidence the learning outcomes. Portfolios do however have obvious advantages in terms of student ownership of their work, and in terms of their ability to record the learning acquired over a period of time where this may vary in pace and depth. Reflective portfolios offer the capacity to be assessed both formatively and summatively. Portfolios allow students to draw on a wide range of different types of academic work and readily accommodate different styles of writing to evidence learning. They do however pose a challenge for students in terms of their ability to manage their workload and learning over time and to sustain their capacity for reflection over the life of the whole module which is usually the selected assessment period. It is not uncommon for law students engaging in the clinic setting who are required to complete a reflective portfolio /log/ to develop ‘reflective fatigue’, feeling obliged to write and add ‘content’ to their portfolio but having no new insights to offer. The danger then becomes one where ‘reflective journals tend all too frequently to remain as fragments held together merely by a chronological sequence.’23 The learning process can become fragmented so that students see their reflective portfolio as a product and outcome of their learning rather than part of the learning process itself. The main criticism of portfolios is the tension between capturing a range of learning outcomes, given that students have to design and deliver the structure to support the attainment of these. Portfolios are most effective when conceived as fusion of process and product24 not solely a product of the learning process, achieved by the process of reflection which contextualises the evidence in the portfolio.25 Moreover students can all too easily reflect continuously but without synthesising the totality of learning from their experiences, focusing on reflections as single 22 Maisch M. M(2003) ‘Restructuring a Master’s degree dissertation as a Patchwork Text’, Innovations in Education and Teaching International, 40: 2, 194-201 23 Scoggins J Winter R, ‘The Patchwork Text: a coursework format for education as critical understanding’ Teaching in Higher Education, (1999) 4 (4) 485 – 499 24 Winsor, P. & Ellefson, B. ‘Professional Portfolios in Teacher Education: An Exploration of Their Value and Potential’ The Teacher Educator, 31(1) (1995) 68-91. 25 Baume D. (2001) ‘A Briefing on Assessment of Portfolios’ Assessment Series 6 (York :LTSN Generic centre) Assessing Law clinic – the use of digital patch text assessment as an alternative to traditional portfolios 622 International Journal of Clinical Legal Education Issue 20 events over the direction of the module. Portfolios then become a mere collection of individual elements selected to evidence learning26 each connected to the same theme but not synthesised to each other and related to the overall learning experience. To use the analogy of a jigsaw in a box, the same elements are present unassembled, but the view is entirely different once the jigsaw is assembled and complete. The difficulty for students in compiling portfolios, relates to the often wide assessment parameters that assessment of law clinics require. Requiring students to demonstrate practical skills development and their understanding of their relevance, value and application in practice through reflection, is a challenge. In addition students also have to select the most appropriate learning events on which to reflect to meet the diverse range of parameters often assessed by law clinic modules. Where portfolios are summative, if students fail to integrate and synthesise their work beyond a collection of individual pieces, they risk not meeting the assessment parameters and ‘missing the mark’. A selection of pieces alone without drawing these together in a synthesised coherent way denies the student the opportunity to fully review and reflect on the totality of their learning, and diminishing the capacity to demonstrate the quality of their learning. This is crucial where clinic modules are assessed, as is often in the final year of study and where the period of time over which the clinic module offered is linked to the provision of credit in a single or even a double credit module. Outline of research aims and methodology This paper aims to outline the participation of the Derby law School in a year long project funded by JISC (Joint Information Systems Committee). JISC is a UK national charity that works to support educational institutions, including those in higher education, to support new developments in information and digital technologies and their application in the higher education sector. The DePTA project – Digitally Enhanced Patch Text Assessment was funded by a learning teaching and innovation grant from JISC. This project aimed to investigate the use of digital technologies to enhance the use of PTA in a variety of higher education environments. The project was a collaborative project involving the participation of five higher education institutions offering a range of different traditional and non-traditional subjects at degree level. The evaluation of the project was undertaken by the Centre for Recording Achievement. The project aims were defined as follows: • To assess the extent, if any, to which digital media could enhance the use of PTA • To extend and assess the relevance and utility of PTA to the more traditional undergraduate subject areas ( such as Law) in different higher education institutions For the purpose of this paper reference is made to the application of the methodology and the findings in relation to the law subject area only and its application within a law Clinic module. Project methodology In total there were five participating institutions, selected on the basis that there ‘is a notable absence of evidence that innovative forms of assessment are being used in the more traditional 26 Baume D. (2001) ‘A Briefing on Assessment of Portfolios’ Assessment Series 6 (York :LTSN Generic centre 623 academic disciplines and institutions’.27 Participating institutions were required to allocate a project partners/ contact person from the academic team who was engaged in teaching /supporting the module and would support the project throughout. The PTA was to be applied to a final year module on an undergraduate degree. The selection of the digital technology was made by the participating institution according to the technology available and that most appropriate to align with the assessment strategy. The design and configuration of the PTA including the number of patches was also determined by the partner institution, the only requirement that all institutions provided assessment criteria for each assessed ‘patch.’ In implementing the project methodology the participating institutions needed to attend to the project objectives. These included: • To extend the use of e-assessment into undeveloped areas i.e. traditional subject areas such as law • To evaluate the extent to which the scope of PTA assessment can be extended to embrace subject benchmarks and graduate skills, in particular difficult-to demonstrate attributes where evidence needs to be captured from learning processes as well as products • To consider the capacity to enhance and evidence student learning at a high-stakes level, and the sustainability in relation to staff workload. The project was evaluated using two approaches, one designed to capture student feedback on the use of PTA within the specific the module to which this applied, using a student questionnaire. This has a number of questions to which the student has to give a rated response (score of between 1 and 5) giving a crude indicator of the depth of agreement with the statement. This was completed at the end of the project and after students had submitted their coursework. This was additionally supplemented within the Law School by a semi-structured interview with student participants, to gain a more in depth student perspective of the value of the module and their participation in the project overall. The second component of feedback was that obtained from the partner institutions, through semi-structured interviews with the project coordinator, to gain their perspective of the project. This was supplemented by a case study video presentation by the project partner, which described in more detail how the project was implemented in that partner institution and briefly outlined some of the findings relevant to the subject area. The project as applied at the partner institution. The project involved the use of PTA in the context of a final year module on an undergraduate degree in subject discipline. The law subject implemented the PTA into the Clinic module, a year long optional module on the final year of a UK qualifying law degree. The Law clinic module is delivered several times a year, cohort sizes being small, given the module was a double credit module and seven students from the cohort of twelve, selected participated in the project. As a final year module, it contributed to 25% of the marks for the final year, with the final year marks comprising 80% of the final degree classification. This made the module ‘high stakes’ for those taking part in the project. 27 Marcangelo C., ‘Jisc Final Report’ 02/08/2011 para 3.2 Available at http://dpta.wordpress.com/an-overview/ Assessing Law clinic – the use of digital patch text assessment as an alternative to traditional portfolios 624 International Journal of Clinical Legal Education Issue 20 The Law clinic module is assessed by reflective portfolios, student reflections additionally supported by evidence demonstrating the student engagement with the learning setting and their participation in and observation of a variety of legal tasks. The evidence may include records of meetings attended, interview observations, client case studies, minutes of team meetings, feedback from placement supervisor / mentor. Additionally anonymised samples of practical tasks such as client letters, statements of claims, undertaken by the student may also be included.28 A detailed activity attendance log was also required as part of the assessed portfolio, requiring students to account for their time, as required of practitioners and is also relevant to voluntary sector accountability for funding. This enables students to be commercially aware and promotes good time management skills. Student portfolios may also include reflections on outcomes of assessment of their competence in key tasks such as drafting. Due to the DePTA project lead time, there was insufficient time to change/ modify the existing module learning outcomes under the University quality assurance processes. One of the challenges was the use of the existing learning outcomes and assessment methodology and the incorporation of PTA within this. It is worth mentioning here that ideally the use of PTA should be undertaken alongside specifically configured learning outcomes rather than using PTA as a ‘bolt on’ to existing learning outcomes. The PTA consisted of 4 patches 1. Critical reflection on: a) an aspect of professional practice b) an aspect of legal process and service from a client’s perspective. 2. Submission of client case studies. 3. Critical reflection of the service area and the role of legal advisors, as to their value to community they serve. 4. Final reflection – this brought together previous PTA tasks and supporting evidence, which was presented as a single piece for assessment. Patches one and three were submissions where the students could view the work of the other students and offer feedback on ‘patches’ submitted by the other students, as well as being able to read the tutor (member of the academic team) comments on the patches submitted. The feedback process for the shared submissions was therefore reciprocal, with the opportunity to both give and receive feedback from their peer group and the module tutor. The shared submissions aimed to enhance the student’s capacity for critical reflection and develop their individual style of reflective writing through encouraging critique of the reflections and writing styles of other students. The requirement to provide peer feedback was not part of the assessment requirements and was entirely voluntary in the sense that the provision of feedback by the tutor was not conditional upon student having provided peer feedback. 28 Ledvinka G. ‘Reflection and assessment in clinical legal education: Do you see what I see?’ (2006) 9 Int’l J. Clinical Legal Educ. 29 ,46 625 The aim of the project was to evaluate the relationship between the use of PTA and digital media and to establish what factors would support, facilitate or enhance any student engagement with PTA and the assessment outcomes. The aim was not to review the digital media as an assessment tool. The digital media selected was the ‘pebble pad’ on-line e portfolio system and virtual learning environment. E-portfolio systems are generally recognised to be ‘an archive of material, relating to an individual, held in a digital format’.29 Pebble pad is similar in structure and presentation to paper portfolios, but potentially offers a broader representation of student work and learning due to the expansive nature and ability to capture, store and present work in different digital forms. The selection of pebble pad was not aimed to reproduce the paper based portfolio in a digital form, but to enable students to share patches electronically at pre-determined dates. Students and tutors create an institutional account within the module. The tutor creates common access areas to enable student submission of PTA and submission and viewing of feedback. The tutors can view all material within the online student portfolio. Pebble pad enables the student to create reflections and daily diaries in a variety of pre-set formats. Students can also upload word documents, scanned images, video footage, audio files, and add weblinks, termed ‘assets’ The student can create a portfolio within the pebblepad platform which allows the tutor to view selected work and assets. The tutor receives an email of the title of all assets uploaded to the student portfolios and any amendment to the portfolio, so can view the pace and extent of student progress and track changes made to the PTA in light of the feedback given. The selection of this digital media was carefully made in consideration of its ease of use and the implications for any student training, for those unfamiliar with this.30 Pebble pad was selected for its ease of use and accessibility and its capacity to enable students to integrate evidence to support their reflections and their learning.31 It also enabled students to submit their PTA online and to make shared submissions and to engage in and view peer feedback. Additionally, one advantage of the pebble pad portfolio system was that students would continue to have access to the pebble pad platform after completion of their degree, which they could use to potentially develop their own CV,32 customising their presentations and access,33 for self-marketing to enhance employability.34 The inclusion of work from the clinic module was only possible due to the constraints of submission which ensured that no material referred to in the ‘patches’ would compromise client confidentiality, being purely generic and reflective in nature. 29 Madden. T., ‘Supporting Student e-portfolios: A Physical Sciences Practice Guide’ September 2007 , 4. 30 Hudson L ‘Techniques’, 79 (5) (2004) 19. 31 Chappell D S and Schermerhorn J R ‘Journal of management education’, 23 (6) (1999) 651 32 Madden. T., ‘Supporting Student e-portfolios: A Physical Sciences Practice Guide’ September 2007,9. 33 Joyes, G., Gray, L. & Hartnell-Young, E. (2009). Effective practice with e-portfolios: How can the UK experience inform practice? In Same places, different spaces. Proceedings ascilite Auckland 2009. http://www. ascilite.org.au/conferences/auckland09/procs/joyes.pdf ‘Learners may create multiple e-portfolio presentations, for a range of purposes, for different audien’ces, at different times. For example, for presenting evidence of skills and achievements to an employer, or presenting reflections on a work placement as part of a course of learning.’ 34 Merrit DJ., ‘Pedagogy, Progress, and Portfolios (2010)25 Ohio St. J. on Disp. Resol. 7, 14 Assessing Law clinic – the use of digital patch text assessment as an alternative to traditional portfolios 626 International Journal of Clinical Legal Education Issue 20 Project Outcomes The overall project findings are limited as student numbers here were small, only seven students out of the twelve participated. Other institutions and subject areas had larger student cohorts. The overall project aimed to investigate the use of digital technology to enhance the use of PTA in more ‘traditional’ undergraduate subjects, but absent any control group using PTA without drawing on the use of digital technology, it has been difficult to isolate the benefits of the digital technology alone other from the participant feedback questionnaires. It was noticeable that students were influenced in their engagement and their willingness to participate in the project, by their response to the use of the digital media. In some instances the digital media was the reason for some students declining to take part in the project. Here the perception being that the time taken to construct the online portfolio within which the PTA sat, would take longer than construction of a paper version. This response may indeed have been different if the PTA had stood alone and not been integrated into the original assessment by portfolio, for this module. The fact that a number of students were adverse to taking part in the project seemed to be related to the selected digital media or the use of PTA as part of an assessment strategy, both of which could be considered to be innovative assessment methods for this student group and outside their experience of more ‘traditional’ assessment methods. The views of students who declined to participate was captured by the student questionnaire. One student commented that they did not like engaging with technology and were put off by the ‘technical stuff.’ Another student commented that they were put off by their previous use of pebblepad describing this as ‘inefficient and slow.’ Two students who did not participate advanced the timing of the project as the reason for this, wishing to focus on their final classification and the pressure of participation generally. For those students who did take part, five when asked stated that they agreed that the use of the pebblepad had had a positive impact on their work, but there were no further comments to illuminate as to the nature of this impact. Students were provided with a two hour demonstration and use of pebblepad. Whilst some responded very positively to the use of the pebble pad, all acknowledged that uploading of anonymised evidence as assets to support their reflective PTA was difficult requiring the scanning of these which was time consuming. The scanning of assets also countered the benefit of accessing their pebblepad electronically at distance from the University, since scanners were only available on campus. The academic environment draws on a variety of learning technologies which students make full use of through the University platform. In contrast there is a more limited use of digital technologies in legal practice, particularly by the small scale legal service providers in which students were situated for their clinic placement. The law profession has a very traditional approach to record keeping, and it remains the case that client files and documents are typically in paper form. The cultural and professional context of the law in practice is still very much geared to paper evidence, claims submitted and case file correspondence being in paper form .This factor was an additional encumbrance and separated the assessment media from the reality of the context of practice. This is however subject specific relating to the context of the practice of law and the module selected for the DePTA project at this institution. One of the other participating institutions selected a photographic design module for which the production of a digital online portfolio was highly relevant to the subject discipline. Whilst the use of digital media is subject specific it remains relevant for promoting engagement 627 with PTA in some subjects. For law however, student feedback confirmed that the digital media facilitated the ‘ease’ of submission and the opportunity to provide and access feedback whilst off campus. The digital media did not however increase student motivation to provide peer feedback. Crucially for this project the digital media did not have a significant impact on the engagement of law clinic students with the PTA. Students stated that they did however gain from reading the PTA submissions of other students, but their engagement with providing peer feedback on this was poor. Further investigation into the poor response to peer feedback, indicated that this was due to not feeling ‘skilled’ in providing feedback, wary about how useful this would be to recipient students and not wanting to appear negative. They also perceived the tutor as adopting the role of ‘expert’ in this regard. The reluctance of students to engage in providing peer review is no doubt borne of their heightened sensitivity to the personal nature of reflections and their content, one of the few occasions where students can legitimately write in the first person in offering personal insights and their ‘feelings’ regarding their learning experiences. Here the use of digital media to provide feedback may make the availability of feedback easier, but may make students uncomfortable with the level of formality. Feedback given in tutorial groups by comparison may feel a more familiar process, can be provided verbally and more informally. This author considers that had the feedback to the patches, in relation to the shared submissions been given in face to face in groups sessions, students’ participation in the project would have been greater an in relation to offering peer feedback. There was however a reliance on tutor feedback which was perceived as valuable, focused and aligned to both the individual PTA assessment criteria and the overall module learning outcomes so was more useful to students. Again the perception of the value of the tutor feedback was relevant to the value of the PTA as a process but the use of the digital media to enhance this is unclear. Students in the final questionnaire were only asked if the use of technology had a positive impact on their work, arguably too broad a question to isolate the role of the digital in enhancing the use of PTA. Five of the seven agreed; stated that it did, but did not illuminate as to what the precise benefits of the digital technology were, and the semi structured interviews did not illuminate further. Students obviously have their own individual perceptions of the value and limitations of the selected digital media in relation to the assessment task and further research needs to be done to investigate this relationship in more depth absent a specific connection to PTA. However balanced against this two students created their own e-portfolios on line which they planned to use in relation to supporting their future job applications and career progression. The appreciation of this potential was a motivating factor in the use of the digital media.35 In the main the feedback from students was that PTA in this instance did not significantly reduce the workload associated with the module assessment from the student or tutor perspective, but neither did it increase workload. What tended to emerge, was students focused more on the assessment brief of each patch and reworking this in light of feedback, rather than on the volume of writing usually associated with portfolios. This was not aim of the project, but the use of PTA did however help students write more concisely, critically and with purpose, so enabling them to work smart. This emerged from the tutors’ previous experience in teaching on the module and marking portfolios, as well as from informal feedback to the tutor during the project and from the student semi-structured interviews at the end of the project. The improvement in quality and 35 Welzer K and Strudler N ., ‘Journal of Research on Technology in Education’, 38 (2) (2005)231 Assessing Law clinic – the use of digital patch text assessment as an alternative to traditional portfolios 628 International Journal of Clinical Legal Education Issue 20 penetration of critical reflection is important to secure the required level of analysis36 associated with final year study. The use of PTA also enabled students to be more clear and confident in their progress, in light of the feedback received on each patch, and to identify early on gaps in their learning and synthesis of this. This was evident to the tutor from their observation of the changes in the quality of student writing with the completion of each patch in response to feedback provided. The author believes this ‘work smart’ effect could be better achieved by the use of word limits for each PTA, which was not a parameter in this project. In this project the PTA was integrated into the existing module assessment and learning outcomes. A more effective and flexible approach would be design the module learning outcomes and the PTA at the same time. From a tutor perspective one concern about the sharing of PTA for the future in its application to other modules is the need to counter any possibility of plagiarism or collusion. This in fact was not an issue here, as the students on the Clinic module all have very different learning experiences deriving from being placed with different placement providers and the uniqueness of their learning experience and environment. This would however be a concern for application to other law subjects where substantive knowledge is the central focus of module assessment. Students did however state that they derived considerable benefit and assistance from the structure that the PTA provided in relation to the assessment strategy and clearly felt more confidence in knowing they were on track in attaining the learning outcomes. PTA process based on a formative assessment model enabled students to make full use of feedback to identify any gaps in their learning, facilitating the structuring of their learning alongside and integral to the reflective process. In particular the submission guidelines which were available to all students on the module, including those not participating in the project, were also valued for their contribution in deconstructing the overall learning outcomes into small assessed ‘pieces’. The guidance illuminated as to the assessment parameters and the relationship between the individual patches, their relationship to each other and the overall learning outcomes. The PTA does appear to encourage active and dynamic use of tutor feedback on patches assessed, which was shown in the number of changes students made to their patches after having received feedback on these. This would not usually occur within the concept of traditional portfolios which were previously summatively assessed. Students in the feedback at the end of the project and their informal interviews, stated that the individual nature and specificity of the feedback motivated them to make use of this in revisiting their current patches and their work on future patches, individualising the learning process.37 All but one of the student participants stated that the feedback they received helped them improve their final submitted work. Students also stated that the feedback was helpful in synthesising theory and practice more effectively, this was also evident in the qualitative changes in their work across the patch text assessment process. The PTA structure and process helped students to link the PTA guidance to the overall learning outcomes to more clearly outline their progress to date. The feedback acted to feed forward to inform future reflective writing. It also focused student attention to improvements needed and how this could be achieved. The underpinning structure of PTA enhanced student autonomy, and as feedback was directed to the reflective process not the context or content. The feedback offered no right or 36 Ledvinka G. ‘Reflection and assessment in clinical legal education: Do you see what I see?’ (2006) 9 Int’l J. Clinical Legal Educ. 29 ,39 37 Waye V, Faulkner M, ‘Embedding E-portfolios in a Law Program: Lessons from an Australian Law School’ 61 J. Legal Educ. 560 2011-2012, 564 629 wrong but was geared to promoting more critical enquiry into the student experience, attitudes and observations and knowledge. Students did however identify that it enhanced their ability to reflect in depth and holistically. The PTA potentially enables students to more effectively synthesise their learning experience and integrate theory and practice promoting what Winter has described as a ‘gradual’ model of learning as ‘making sense over time’ lies at the heart of the PTA.. the sequence of tasks within ..is intended to build into the assessment process a recognition of learning as a gradual ‘coming to know’38 This resulted in what this author regards as an active and upward spiral/ cycle of learning along the life of the module with students motivated by the feedback they received and keen to apply this to future work. The motivational aspect of PTA and its alignment with the learning process in maximising the use of formative feedback has an obvious advantage over the traditional portfolio type of assessment. Unfortunately the findings do no indicate a relationship between the digital media used and the benefits arising from the use of PTA, other than student agreeing that the digital media had a general ‘positive impact’ on their work. Conclusions – Future use of PTA The use of PTA in assessing CLE offers a number of benefits also recognised in other learning contexts.39 It provides a framework for the provision of feedback to students against clear assessment guidelines. This enables student to maximise the potential use of feedback for the future since this is linked to the process of learning and critical thinking and the process of reflection. This reduces concerns in providing feedback related to ‘content’ where tutors may have concerns about the their feedback, and the balance between assisting the learning process and feedback filling the gaps in student knowledge directing students to the ‘answers.’ Assessment guidelines when provided offer the means for students to link the requirements of each ‘patch’ to each other and the overall assessment goal, reconstructing the broad learning outcomes and the process of assessing these, to enable students to more effectively meet these. It also assists the learning process since PTA is crucial to enable students to maximise learning opportunities directed by the PTA assessment process.40 This helps students keep on track and avoid ‘portfolio drift’ whereby student continue to add pieces to their portfolio without consideration of their value or contribution. However it remains the case for this project that the disadvantage of portfolios could also have been overcome by the provision of clearer guidance in their structure. The real gain in PTA is the structuring of the assessment process and its capacity to drive the learning process through the use of feedback to develop the skill of reflection, targeted to more narrow assessment parameters, one patch contributing to the larger whole. All the work the students do is perceived as relevant in contributing to the final product, important given that 38 (n25) 39 Rees , B. and Preston, J (2003) The International Patch work: Introducing Reflexivity into the Business Curriculum Innovations in Education and Teaching International Vol. 40 (2) 123-132. 40 Khan, P (2006). Supporting reflective processes with students: insights from a review of research, available at: http://www.heacademy.ac.uk/assets/York/documents/ourwork/research/literature_reviews/reflective_practice_ with_students.pdf Assessing Law clinic – the use of digital patch text assessment as an alternative to traditional portfolios 630 International Journal of Clinical Legal Education Issue 20 students are increasingly focused on assessment tasks. In addition if each PTA is supported with assessment guidance as well as formative feedback on completion then the PTA process ‘feeds’ forward to assist future learning and development closing the ‘feedback loop’41 through the resulting action. This maximises the opportunities for future learning through reflection since each PTA can be amended up and until the final submission with all patches being incorporated into the final ‘patch.’ These qualities are often not incorporated into traditional portfolios which focus on the form of content, including reflections, sample case study, case conference meetings, rather than the process of reflection linked to these.42 Previous studies have shown that the use of portfolios, in particular eportfolios such as pebblepad, without the embedding of the PTA within them, as was the case in this project, focus students attention to the construction of the portfolio as to content, without being accompanied by the relevant learning.43 In the case of the Derby law clinic, the use of PTA as a replacement for the traditional portfolio has much to offer for the future, even without the use of digital media to support this, since the findings of the project revealed that the selected digital media itself did not appear to add value to the overall assessment or learning process. The impact of PTA on learning may have been more apparent had the PTA been specifically designed alongside the learning outcomes rather than grafted onto these. The engagement with PTA may have varied with the selection of a different form of digital media, or indeed allowing students to select the form they felt comfortable with. For the future the author is keen to explore the use of PTA as an alternative to portfolios given some of the advantages they have yielded within this project. PTA offers a structured form of assessment that can potentially be applied to other law modules. However such an application needs careful consideration as to its application to other law modules as to the number of patches, student numbers and time available for providing feedback. The sharing of patches for assessments sharing the same substantive content/ knowledge may need consideration regarding plagiarism and collusion. The DePTA project allowed students to share their unique ‘clinic’ experience and reflections on these, the substance of which varied for each student and so avoided any concern arising over the originality of the student’s work The provision of generic feedback can overcome some of these concerns, students in this project valued the individualised nature of their feedback which was manageable for tutors given the small size of the student cohort. The provision of pre-assessment guidance and post submission feedback can assist learning but a careful balance is required in maintaining student autonomy and encouraging students to take a lead role in the assessment process so reducing tutor dependence in this respect. PTA is an innovative form of assessment which has the potential to enable students to learn and work effectively where the learning outcomes and assessment strategy are creatively designed and aligned to promote this. It potential as an assessment strategy is directly linked to the context of learning and assessment. The design and aims of PTA align well to the experiential model of 41 Hounsell, D.,McClune, V.,Hounsell, J. & Litjens, J. (2008) The quality of guidance and feedback to students. Higher Education Research and Development, 27(1), 55-67. 42 Ledvinka G. ‘Reflection and assessment in clinical legal education: Do you see what I see?’ (2006) 9 Int’l J. Clinical Legal Educ. 29 43 Waye V, Faulkner M, ‘Embedding E-portfolios in a Law Program: Lessons from an Australian Law School’ 61 J. Legal Educ. 560 2011-2012,576 631 learning, which had the most significant impact on learning. The use of digital media to enhance PTA is however more sensitive to the profession and practice norms, which seemed to operate as a barrier to engagement with PTA here, rather than a trigger to facilitating this. Student attitudes to the mode of assessment are very much influenced by their previous experiences and understandings44 and their confidence in any digital media used in this process. The concept of patch text assessment aligns well to assessment strategy of law clinics, in promoting concise and incisive reflection on experiential learning. It also provides a structured assessment process that is aligned to capture and promote the development of personal and legal skills and substantive knowledge acquired from practice settings, which the essence of the law clinic experience. Karen Clubb University of Derby IJCLE Conference Durham, July 2012 44 Marcangelo C., ‘Jisc Final Report’ 02/08/2011 para 2.3 Available at http://dpta.wordpress.com/an-overview/ Assessing Law clinic – the use of digital patch text assessment as an alternative to traditional portfolios 632 International Journal of Clinical Legal Education Issue 20