Evaluation of Collaborative Assessment of Work Integrated Learning Judith McNamara and Elizabeth Ruinard1, Queensland University of Technology, Australia INTRODUCTION The international trend towards an increasingly standards-based approach to higher education and the resultant focus on the assurance of learning in tertiary programs have generated a strong emphasis on the assessment of outcomes across the higher education sector. In legal education, curriculum reform is highly prevalent internationally as a result of various reviews of legal education, including the publication in 2007 of the report by the Carnegie Foundation, Educating Lawyers: Preparation for the Profession of Law, and more recently, the 2013 Review of Legal Education and Training (LETR) in England and Wales. The report of the LETR included a recommendation to: “enhance consistency of education and training through a more robust system of learning outcomes and standards, and increased standardisation of assessment” (Legal Education and Training Review, 2013). The shift in focus of legal education has resulted in a reconsideration of the way in which outcomes are assessed. Traditional assessment in legal education involves a mix of essays, case notes, problem solving tasks, research assignments and examinations which have targeted assessment of the understanding of the core areas 1 Judith McNamara is the Assistant Dean, Learning and Teaching in the Faculty of Law at QUT. Elizabeth Ruinard is Learning and Teaching Developer, Law and Health at QUT. ( Special Issue Problematising Assessment in Clinical Legal Education ) ( 10 ) of legal knowledge and legal reasoning skills. More recently assessment techniques have been broadened to include a range of instruments, such as role plays and simulations, for the assessment of legal skills. These assessments may lack authenticity in that they are decontextualised, restricted to defined knowledge, tasks and settings, and are subject to other constraints such as time limits in examinations (Hughes, 2009). As legal education focuses more on the attainment of a broader set of outcomes encompassing soft skills, capabilities and attributes, more authentic assessment will need to be developed appropriate to this new environment, meaning that modes of assessment with strong application in real-life settings should be preferred. In designing new assessment frameworks, legal educators can draw upon the body of literature around the assessment of professional competence in medicine and other professional education. Professional competencies in the context of medicine are well defined through a multi-dimensional model encompassing a broad range of knowledge, skills and attributes including soft skills, professionalism and meta- competencies (Epstein and Hundert, 2002). The existence of these competencies has driven more innovative approaches to medical education and assessment of outcomes (LETR, 2013, p.142). For example, a commonly used assessment technique in medical education is the objective structure-clinical examination (OSCE) which is a “form of practical, usually simulation-based, assessment” (LETR, 2013, p.142). This form of assessment corresponds to “showing how” in Miller’s model. In the model there is a movement upwards and increase in complexity from the cognitive “knowing” and “knowing how” to the behavioural “showing how” and “doing”. Figure One – a simple model of competence From: www.faculty.londondeanery.ac.uk/e-learning/workplace-based-assessment/what-is- workplace-based-assessment The assessment of medical undergraduates (and legal undergraduates) has tended to focus on the triangle base: “knows” – i.e. the straight recall of knowledge; and “knows how” – the application of knowledge to problem-solving and decision- making. Assessing “shows how” is challenging but achievable through OSCE in the medical context. Nonetheless with OSCE validity risks being lost at the expense of reliability, since complex skills, requiring an integrated professional judgment, become fragmented by the relatively short length of time assessors are able to spend at each station (Van der Vleuten, Shatzer and Jones, 2001, p. 646). The real challenge lies in assessing a student’s actual performance on the wards/in the consulting room (or with the client/in court). Composite medical examinations and portfolio assessment have been recommended to assess “doing” in the medical context although this can be time-consuming and costly (Van der Vleuten, Shatzer and Jones, 2001, p. 649). In the light of the shift in emphasis in legal education to a more outcomes-based approach, the unique capacity of experiential learning in law, including clinical legal education, to contribute to and enliven the development and assessment of outcomes has come to prominence. Experiential learning provides “rich contexts” for the implementation of more authentic forms of assessment (Hughes, 2009). Assessment of clinical legal education (CLE) has unique insights to offer in relation to ways in which the legal curriculum might develop robust and academically accepted ways of assessing competence more generally. As new assessment instruments are developed it is indispensable that they be evaluated to ensure they satisfy the basic principles of assessment such as validity, reliability and fairness. In this regard Van der Vleuten proposes a “utility model” offering a framework for the evaluation of assessment instruments. The model is said to be useful in helping “educators make considered choices in selecting, constructing and applying an assessment instrument” (Van der Vleuten, 2005, p. 310). The paper will foreground the advantages of work-integrated learning (WIL) for the assessment of professional judgment and demonstrate how such an impetus accords with Van der Vleuten's approach to assessment. WIL is defined as an “umbrella term for a range of approaches and strategies that integrate theory with the practice of work within a purposefully designed curriculum” (Patrick, Peach et al 2008, p. iv) and subsumes CLE and other types of work-based, experiential learning. The paper proceeds to explain the distinction between the learning outcomes versus professional competencies curricula, where WIL belongs to the latter and where WIL assessment has strong potential to incorporate multiple viewpoints and be discerning about the development of the student’s professional judgment. Part of the WIL assessment approach will be articulated with Van der Vleuten's position on validity, reliability and educational impact, with WIL being well-placed for demonstrating emerging professional judgment because of the strong dimension of reflection and reflective writing on the WIL learning experiences which occurs therein. LEGAL EDUCATION IN AUSTRALIA In Australia, the advent of the new standards-based regulation of the higher education sector, including the Australian Qualifications Framework, and the Threshold Learning Outcomes for Law, are key drivers for reform. Since the 1980s, Australia has gradually shifted to a more outcomes focussed legal education regime. While the principal requirement for the academic qualification for admission to legal practice remains the prescribed areas of knowledge known as the ‘Priestley 11’, this is supplemented by the regulatory framework for higher education incorporating the Threshold Learning Outcomes for Law. The educational requirements for admission as a legal practitioner in Australia consist of an approved academic qualification and practical legal training. The academic requirements are constituted by an approved course of study representing at least three years full-time study of law and a satisfactory understanding and competence in the prescribed areas of knowledge.2 There are eleven prescribed areas of knowledge: criminal law and procedure, torts, contracts, property, equity (including trusts), company law, administrative law, federal and state constitutional law, civil procedure, evidence and ethics and professional responsibility.3 Generally the course of study is a university Bachelor’s degree, Bachelor Honours or Juris Doctor. As tertiary qualifications, such courses are regulated by the Australian Qualifications Framework (AQF) which provides a comprehensive, nationally consistent but flexible framework for all qualifications in post-compulsory education and training in Australia. Comprising fifteen qualifications, ranging from Certificate I to Doctorate, the AQF specifies the relevant skills, knowledge and application of skills and knowledge as well as volume of learning for each qualification. The AQF Guidelines articulate the main criteria for defining qualifications based on the specific characteristics of education and training at each qualification level. These characteristics are expressed principally as learning outcomes. Law qualifications in Australia are typically either level 7 Bachelor, level 8 Bachelor Honours or level 9 Juris Doctor. 2 For example refer to rule 6 Supreme Court (Legal Practitioner Admission) Rules 2004 (Qld). 3 For example in Queensland see Attachment 1 to the Supreme Court (Legal Practitioner Admission) Rules 2004 (Qld). For example, Bachelor Honours degree qualifications must be designed and accredited to enable graduates to demonstrate the learning outcomes expressed as knowledge, skills and the application of knowledge and skills specified in the level 8 criteria and the Bachelor degree descriptor. Graduates at this level will have advanced knowledge and skills for professional or highly skilled work and/or further learning. Knowledge: Graduates at this level will have advanced theoretical and technical knowledge in one or more disciplines or areas of practice. Skills: Graduates at this level will have advanced cognitive, technical and communication skills to select and apply methods and technologies to: · analyse critically, evaluate and transform information to complete a range of activities; · analyse, generate and transmit solutions to complex problems; and · transmit knowledge, skills and ideas to others. Application of Skills and Knowledge: Graduates at this level will apply knowledge and skills to demonstrate: · autonomy, well-developed judgment; and · adaptability and responsibility as a practitioner or learner. In addition to complying with the descriptors for the relevant qualification, the outcomes for the qualification must reference the Threshold Learning Outcomes (TLOs) developed for the discipline of law and implemented in 2013. The TLOs were developed by discipline scholars appointed by the national Office of Learning and Teaching and are defined in terms of minimum discipline knowledge, discipline- specific skills and professional capabilities, including the attitudes and professional values expected of a graduate from a specified level of program in a specified discipline area. One set of TLOs pertains to both level 7 and 8 qualifications whilst a separate but comparable set exists for level 9 qualifications. The TLOs for level 7 and 8 qualifications comprise TLOs including: TLO1 Knowledge, TLO3 Thinking Skills, TLO4 Research Skills and TLO5 Communication and Collaboration but for this discussion the focus falls particularly on TLO2 and TLO6. Typically law schools in Australia have developed and articulated program learning outcomes which reference, incorporate or in some cases directly mirror the TLOs. For the purposes of this paper, the TLOs will be treated as if equivalent to program learning outcomes. Different approaches might be taken however, in accordance with the principles of whole-of-course design, and in order to provide assurance of learning, the TLOs would usually be developed throughout the course and mapped to assessment in individual units. In addition to the academic requirements, an applicant for admission to legal practice must also have completed the practical legal training requirements. The completion of an award which includes the Competency Standards for Entry-Level Lawyers, along with a minimum of fifteen days supervised experience in a law or law-related work environment, serves to fulfil the practical legal training requirements. The prescribed competencies comprise Skills (lawyer’s skills, problem solving, work management and business skills, and trust and office accounting), Compulsory Practice Areas (Civil Litigation Practice, Commercial and corporate practice, and property law practice) Optional Practice Areas (any two of various practice areas) and Values (ethics and professional responsibility). Each practice area includes a number of specific descriptors in addition to a number of elements for which relevant performance criteria are defined. These criteria in turn list specific tasks that the student must be able to perform in order to demonstrate competencies. Accordingly, it is apparent that in Australia, there is an epistemological divide between the assessment of outcomes for the purposes of academic qualifications in undergraduate law and the assessment of specific competencies, broken down into specific tasks in the Graduate Diploma in Practical Legal Training, completed after the undergraduate qualification. In the Australian system legal clinics and other WIL subjects such as externships are generally completed in the undergraduate qualification. The placement component of the Practical Legal Training (PLT) is largely assessed on a pass/fail basis upon completion of the required hours rather than the demonstration of specific competencies or outcomes, unlike undergraduate WIL placements. This renders the assessment approaches for practical legal training somewhat at odds with Van der Vleuten’s recommended model, which advocates the assessment of integrated competencies. The whole-of-task approach is foregrounded in the competency emphasis presently receiving endorsement. Assessment in WIL is particularly wont to capture the performance of integrated competencies through the demonstration of whole tasks or a series of associated tasks and evidence of associated judgments made and attitudes revealed. WIL assessment is also liable to encode the perspectives of multiple assessors in the workplace, utilise different weightings of criteria, negotiated criteria and a more ‘qualitative’ approach than is available in other contexts, as per the above model (Van der Vleuten and Schuwirth, 2005). WIL IN LAW WIL implicates learning in three domains: learning theory (understanding how to learn), critical reflection and capability (Brodie & Irving, 2007). Capability involves transferrable skills and know-how, and discipline specific knowledge and skills, essentially, professional competence. “Capability” is used here to signify the ability to apply different professional skills and knowledge in the workplace in a general sense rather than a particular sense. Given these components of WIL, the relevant TLOs that might particularly be assessed in WIL include: · TLO 6(b) Reflect on and assess their own capabilities and performance, and make use of feedback as appropriate, to support personal and professional development. · TLO 6(a) Learning and working independently; · TLO 2(d) A developing ability to exercise professional judgment. While the capability outcomes learned and assessed in WIL might include a range of knowledge, skills and professional judgment (Maurer and Cole, 2012), WIL is particularly well placed to assess professional judgment because it “can offer an efficient method of teaching students about professional values and identity essential to becoming effective lawyers” (Maurer and Cole, 2012, p. 143). TLO 2(d) requires law graduates to be able to demonstrate a developing ability to exercise professional judgment. Professional judgment generally has been defined as the “ability to use knowledge, skills and judgment to perform effectively in the domain of possible encounters in professional practice”.4 According to the commentary on the TLOs, it includes ‘the application of knowledge, skills and professional values to serve the interests of clients, justice, the profession and the public good’ and ‘an understanding of the consequences of professional decisions’.5 This current investigation is thus particularly concerned with the assessment of the developing sense of professional judgment in general in the student and even though this has chiefly been interpreted to relate to ethics and professional responsibility (e.g. Evers, Houston and Redmond, 2011), the approach adopted here is to consider professional competence more generally. Professional competence includes the exercise of professional judgment, discretion and reasoning in the application of knowledge and skills in a professional context. It is posited in this paper that professional competence in this sense cannot be dissected into a series of knowledge propositions or professional skills; professional competence is dependent on the understanding of the importance of the context in which knowledge and 4 Kane, MT, 1992. The Assessment of Professional Competence, SAGE. 163-182. 5 Kift , S, Israel, M & Field, R. 2010. Threshold Learning Outcomes for the LLB. ALTC. skills are applied and requires the exercise of judgment and discretion (Cooper and Ord, 2014). The importance of experiential learning in the development of professional competence is highlighted by the Carnegie Report, which suggests that legal education “should seek to unite the two sides of legal knowledge: formal knowledge and experience of practice” (Sullivan, Colby, Wegner, Bond and Schulman, 2007, p. 8). It has been advanced that clinical legal education and experiential learning are the most favourable environments for students to learn about and practise professional judgment. The Good Practice Guide for the teaching of TLO 2 Ethics identified as areas for further investigation (Evers, Houston and Redmond, 2011): · best practice for learning and teaching professional judgment, including clinical and experiential legal education; and · the design of effective feedback and assessment methods for determining students’ developing ability to exercise professional judgment. WIL ASSESSMENT Assessment practices in WIL might be said to be more concerned with assessment for learning, than is more traditional assessment which focuses on assessment of learning. This diverges from traditional legal assessment such as essays and examinations which effect the assessment of knowledge of legal doctrine and theory and where law is taught in a traditional academic environment. Such a situation offers limited opportunity to assess the application of knowledge and skills in a professional context (Hewitt, 2008). In contrast to traditional forms of assessment, WIL assessment tends to highlight the centrality of the learner as an active participant in the assessment process. Assessment is critical to how students make sense of their experience, elevating the learning experience from considerations of process or the application of specific knowledge and skills, to the understanding and exercise of professional judgment. This centrality of the learner is evident in common forms of assessment in WIL which include learning plans, reflective journals, reports, student presentations, classroom discussions (or “rounds”), oral questioning, portfolios, supervisor’s assessment and career plans. Assessment of performance in the workplace can occur through observations, extracted examples of performance of workplace tasks, and various forms of simulation. In WIL it is difficult to predict the learnable moments that will present during the experience, and accordingly, assessment of WIL is generally holistic, focusing on the development of the student’s level of self-understanding, efficacy in the workplace, and awareness of career options rather than on the attainment of particular knowledge or skills (Bates, 2003). After Sylvester, the legal clinic’s context is unique in that it uses a “real client/real emotions, has an unknown dynamic/ changing and evolving factual perspectives, has an unknown outcome/uncertain content and is delivered through a distinctive working relationship with a supervisor” (2015, work- in-progress). With this dynamic in mind this paper focusses on assessment of professional competence in the clinic generally rather than on the specific knowledge and skills that may be developed during the experience. While these may be incidentally assessed, the key emphasis of the assessment falls upon the student’s individual learning strategies and their transformational learning through the reflective process. (These relate to learning to Miller’s “showing how” and “doing”). Despite the student-centred nature of the various WIL assessment methods, they nevertheless might be limited in assessment of professional competence where they rely on the student’s own claims of learning, rather than demonstrated competence (Brodie and Irving, 2007). Engaging the supervisor in the assessment can provide a direct assessment of professional competence. However it is important that where a workplace supervisor is involved in the assessment process they clearly understand what it is they are being requested to assess and that well-defined criteria addressing the required learning outcomes are developed. An emerging interest in collaborative assessment which combines input from the student, workplace supervisor and academic supervisor seeks to find alternative ways of involving supervisors in the assessment process. Collaborative assessment involves the active participation of both the student and the workplace supervisor in the assessment, in addition to the academic supervisor. For example, Zegwaard, Coll and Hodges (2003) propose a framework for workplace assessment mediated by academic supervisors and workplace supervisors. Bates, Bates and Bates (2007, p. 127) suggest that: “University and workplace staff should also supervise student assessment collaboratively, negotiating the detailed requirements with each student and ensuring that appropriate personal reflection on the experience has occurred”. According to Ram, 2008, the use of a portfolio assessment which requires students to provide evidence of learning is a means of supplementing collaborative assessment to ensure that the learning outcomes of WIL are accurately assessed. It is recognised, however, that there can be some limitations to portfolio assessment. Portfolios also have the advantage of fostering learner-centred education and active learning as the students take on their own learning responsibility and effectively manage their own learning. Other advantages include the easily shared dimension of electronic media (in ePortfolios) which enable the students’ learning to reach a wide audience in a meaningful way. In addition, the program of learning for students is evaluated in ePortfolio using pre-determined criteria, thus obliging students to devise a specific plan and generally adhere to the plan (Tosun and Baris, 2011, 47-8)). Further, the ability to present oneself in a professional manner, which the usage of portfolio affects, is an important skill to be acquired by the emerging professional. Some of the disadvantages of portfolios and ePortfolios, however, carry the risk that if academics do not model, direct and support the students sufficiently in learning how to reflect, the students tend to find this process overly challenging and come to resist reflective assessment whenever possible, thus failing to develop adequate reflective skills with sufficient confidence (p.48). Formal assessment of reflection is recognised as contributing to a more profound learning experience for students, raising what might otherwise merely be considered to be work experience to a transformative learning experience from an academic point of view. The author has previously proposed a collaborative model for the assessment of WIL that is reliant on evidence from a mix of sources to ensure professional competence is assessed. The assessment model proposed was: a placement plan individually negotiated between the academic, student and supervisor; a student portfolio or journal which includes student assertions as to capability and direct evidence of work undertaken in the placement, and a supervisor’s report. More recently Cooper and Ord, 2014, have proposed a collaborative assessment implicating a three-way critical review of practice which focusses on the planning, delivery and evaluation of a specific project undertaken by the student during the placement. The utility model suggested by Van der Vleuten provides a framework within which to evaluate the collaborative model of assessment. UTILITY MODEL The utility model proposed by Van der Vleuten (1996) holds that methods of assessment of competence can be evaluated using a framework to weigh the utility of the assessment method according to certain criteria: validity, reliability and educational impact. The framework also implicitly addresses two further variables, acceptability and cost/practicality. The model was developed in the context of assessment of clinical competence in the health sciences. Competence as referred to by Van der Vleuten designates an “aggregate of different components or latent attributes” where expertise in a component allows a person to act professionally regardless of the particular nature of the situation or circumstances. (1996, p. 42) For the purposes of this paper, professional competence refers to the emerging exercise of professional judgment, which cannot necessarily be fragmented into specific, demonstrable competencies. In this regard, competence as defined by Van der Vleuten and assessed in the health sciences may be more closely aligned to the particular competencies which are the domain of practical legal training than it is to the intellectual competencies and emerging professional judgment that more appropriately belong in the domain of the undergraduate law degree. However, this distinction only serves to heighten the importance placed by Van der Vleuten on assessment being holistic rather than being reduced to assessment of the component skills and knowledge that students are required to perform. After Van der Vleuten, it is important to verify that those assessment approaches and instruments adopted are characterised by validity, reliability and educational impact or consequential validity, acceptability and feasibility (Messick cited in Van der Vleuten and Schuwirth, 2005, p. 314). VALIDITY Expressed simply, validity of assessment refers to whether “the assessment measures what it purports to measure” (Hewitt, 2008, p. 145). An assessment method might be shown to be useful if results of the assessment correlate highly with subsequent student performance (Van der Vleuten, 1996, p. 51). Van der Vleuten acknowledges the deficiencies in much of the research in relation to assessment validity; however trends are emerging from the literature. Studies reveal an unexpectedly high correlation between different methods of assessment, e.g. between free response tests and multiple choice questions. (See also Driessen, E., Van der Vleuten C. and Van Berkel, H., 1999) It is contended that the content of the assessment is more relevant to the validity of the assessment than the format of the assessment (p. 51). For example, the validity of a multiple choice quiz is not fixed but depends on the content of the questions. Further, particular assessment types might be more valid in measuring some outcomes than others. Van der Vleuten posits that “what is being measured is not dictated by the method but rather what is put into the method” (p. 51). In the context of a portfolio assessment, it might be argued that the validity will be closely linked to the assessment encoding precise task descriptions and specific criteria for assessment. If the assessment is of specific skills or capabilities then these would need to be the specific criteria for the assessment. More general criteria will not result in the assessment of specific skills and capabilities. Similarly in relation to a supervisor’s report, if specific skills or capabilities are not specified, the supervisor’s assessment of competence in the work placement will not provide any measure of any particular outcome. Van der Vleuten warns against breaking capabilities down to behavioural components in order to promote objectivity as this may lead to the assessment instrument not assessing what is intended as it will not reflect the complexity of the skill being assessed (p. 51). Accordingly, in order for the WIL collaborative assessment model to be considered to be valid, it would be a pre-requisite that the particular outcomes being assessed are specified, either in the subject learning outcomes, or negotiated in the placement plan. The criteria for the portfolio and the supervisor’s report would then need to refer specifically to these outcomes. In the proposed law WIL assessment model the outcome being assessed is professional judgment rather than particular skills or knowledge and as such it may not be necessary for specific outcomes to be established. It will be necessary, however, to be explicit in establishing what is meant by professional competence, and the criteria and standards that must be met. The need for consistency is paramount. A further issue that might impact on the validity of collaborative assessment is that the assessment might arguably be assessing the ability of the student to articulate professional competence rather than the demonstration of competence. However, as argued by Cooper and Ord, 2014, the ability to articulate one’s competence is more important than merely being competent. In this regard the "think aloud interviews" proposed by Krieger and Martinez, 2012, call for assessment of experiential learning that focusses primarily on reasoning rather than performance. Inspired by the medical domain's "think aloud" protocol, this experimental assessment method has been developed to identify the different kinds of cognitive processes used by students as they solve problems in practice. According to such an approach, students in a clinical program are allocated a hypothetical problem that is typical of work they have undertaken in the program. They are then recorded as they talk through the problem, with the hypothesis being that by prompting students to talk about a problem without a filter, a great deal can be ascertained about what they are thinking “in practice”. (In certain domains, however, “think aloud” is only used for research). RELIABILITY Assessment can be said to be reliable if it is “objective, fairly administered, and consistently marked” (Hewitt, 2008, p.145). In the field of the health sciences and many other disciplines, assessment of professional competence has been found to present reliability issues demonstrated by variable performance of candidates across tasks. The reliability of assessment is said to increase with the number of items being assessed; assessments that contain only a “small sample of items … produce unstable or unreliable scores.” (Van der Vleuten, 1996, p. 48) Further, the reliance on a single assessor is also said to reduce reliability; reliability is increased where various assessors are used for each item of assessment. Van der Vleuten suggests that clinical ratings used in clerkships in medical schools are “hopelessly unreliable” (1996, p. 49) as they are based on unstandardised performance and are not on direct observation. Other issues impacting on reliability in WIL are the close relationship between the assessor and the student, and the need to assess performance over an extensive period in the past. For these reasons, the reliability of the workplace supervisor’s assessment in the WIL law model might be questioned. The portfolio assessment is intended to address this issue; the notion of evidence from a mix of sources resembles Van der Vleuten’s support of sampling of a range of assessors’ professional perspectives on the item being assessed. Further, Cooper and Ord’s study indicates that the provision of relatively detailed grading criteria supports the supervisors in making reliable assessments of the students’ performance. However, it may not be reliable if the samples of work provided are not sufficient to disclose the student’s capability in the workplace, particularly if the outcomes assessed are broad and not specific. The issue of reliability of a participatory collaborative assessment in a professional placement is examined by Cooper and Ord. The study concluded that the reliability of the self- assessment and supervisor assessment was improved by the use of more detailed standardised criteria. There is some discussion about inter-rater reliability or a measure of reliability used to assess the extent to which different raters agree in their assessment decisions in this study. Mostly, however, it is the phenomenon of proportionately higher marks being globally awarded to critical reviews as opposed to those allocated to essays which is more intensively emphasised by these authors with reasons for this being suggested (Cooper and Ord, 2014, p, 524). The issue of reliability may not be as of much a concern in the assessment of outcomes in an undergraduate program, which is not assuring attainment or particular competencies, however it remains an issue to be weighed in evaluating the assessment model. Hewitt, 2008, argues that subjectivity is an issue in any skills assessment because of the degree of subjectivity that is inherent in the assessment process. While explicit marking criteria which break skills down into specific components can improve reliability, this strategy has the drawback of trivialising and atomising the complexity of the skills being assessed (Van der Vleuten, 1996, p. 51). However, as Cooper and Ord demonstrate, it is possible to design explicit criteria which retain the holistic assessment of professional competence. Further collaborative assessment which engages all three parties in the WIL relationship actively participating in grading improves validity as it is not limited to the exercise of judgment by a single marker. EDUCATIONAL IMPACT AND ACCEPTABILITY; FEASIBILITY The Van der Vleuten model also includes consideration of educational impact or “consequential validity” (Van der Vleuten and Schuwirth, 2005, p. 314); given that assessment drives learning, the impact of assessment on learning should be considered (Van der Vleuten, 2005). It might be argued that the WIL collaborative assessment model addresses this factor positively because it is based on Biggs’ constructive alignment theory. Hence learning activities and assessment tasks are designed to align to the learning objectives of the subject. The authors acknowledge, however, that there is a dearth of literature sharing such insights and suggest that this might be related to the near-impossibility to: “study the impact of assessment on learning without knowing about the context of the assessment” (Van der Vleuten and Schuwirth, 2005, p. 314). The provision of feedback to students on their performance in the placement is another significant educational matter. In this instance, Stuckey et al (2007) argue that recording student performance, providing prompt feedback and training students to receive feedback are key principles that should be met by WIL in law. Involving the supervisor in the assessment is a means of ensuring that feedback is provided. However, the need to provide regular feedback throughout the WIL experience also needs to be addressed. This might be an issue in relation to the overall design of the WIL subject rather than necessarily an assessment issue. Acceptability, an associated concept, is where students’ perceptions of the assessment process are positive and where they believe that the assessment has been conducted according to the stated procedural guidelines; they have obtained valuable insight into their current level of attainment and they have received useful feedback as to how to rectify their shortcomings and enhance their strengths (McKinley, Fraser, Van der Vleuten and Hastings, 2000, p. 574). An issue emerging in relation to acceptability is the common feedback from students about the difficulty they frequently experience in regard to carrying out reflection. Feasibility refers to the quantum of assessment and assessor training deemed sufficient and necessary to facilitate the conduct of a valid and reliable assessment at the relevant level, together with the provision of structured verbal and written feedback on student performance, with specific prioritised strategies for improvement which students perceive to have high educational impact. Feasibility is therefore what is considered reasonable and cost-effective to meet the purpose of the assessment. (McKinley et al, p.578). It is acknowledged that more could almost always be done but that it is necessary to put limits somewhere. CONCLUSION The current international trend towards a more outcomes based approach to legal education has prompted legal educators to reconsider assessment and other educational practices more generally. The investigation of a kind of epistemological divide between assessment of learning (e.g. in the LLB) and assessment for learning (eg WIL in particular), with practical legal training sitting perhaps somewhere in the middle, may lead to progress in this regard. Assessment techniques currently utilised in WIL in legal education and other disciplines suggest possible approaches that are more focussed on the assessment of outcomes or capabilities than other more traditional methods. Despite the innovative approach taken in assessment in WIL in law, there has been limited research into the effectiveness of such assessment to date. The utility model proposed by Van Der Vleuten provides a positive framework within which to evaluate assessment practices in order to provide continual improvement both in the assessment of WIL and of other aspects of legal education. The application of the model to an existing assessment approach in a WIL subject in law suggests that further refinement of assessment could lead to improvements in assessment validity and reliability as well as impacting positively on the educational impact of the assessment, its acceptability, cost and feasibility. Exploring issues related to assessing the developing sense of professional judgment and professional competence in the student, CLE offers unique models of assessment that might also be adapted to the legal curriculum more generally to unite the dimensions of discipline knowledge and the experience of practice. The augmentation of reflective processes in both realms might further make a positive contribution to the holistic development of the legal practitioner through the various aspects of Australian legal education. REFERENCES Bates, M. (2003). The assessment of work integrated learning: Symptoms of personal change. Journal of Criminal Justice Education, 14, 303-326. Bates, A., Bates, M., & Bates, L. (2007). Preparing students for the professional workplace: Who has responsibility for what? Asia-Pacific Journal of Cooperative Education, 8(2). Brodie, P., & Irving, K. (2007). Assessment in work based learning: investigating a pedagogical approach to enhance student learning. Assessment & Evaluation in Higher Education, 32(1), 11-19. Cooper, S., & Ord, J. (2014). Developing ‘know how’: a participatory approach to assessment of placement learning. Journal of Vocational Education & Training, 66(4), 518-536. Cooper, J. (1991). What is Legal Competence? The Modern Law Review, 54(1), 112-121. Driessen, E., Van der Vleuten, C. and Van Berkel, H. 1999. Beyond the multiple-choice v. essay questions controversy: Combining the best of both worlds. The Law Teacher, 33 (2), 159-171. Epstein, R. M., & Hundert, E. M. (2002). Defining and assessing professional competence. JAMA, 287(2), 226-235.Evers, M., Houston, L., & Redmond, P. (2011). Good Practice Guide (Bachelor of Laws): Ethics and Professional Responsibility Threshold Learning Outcome 2. Retrieved from http://disciplinestandards.pbworks.com/w/file/fetch/52817980/GPG%202%20Ethics.pdf Hewitt, A. (2007). A critique of the assessment of professional skills. Legal Education Review, 17(1), 143-158. Hughes, C. (2009). The Modification of Assessment Task Dimensions in Support of Student Progression in Legal Skills Development. Legal Education Review, 19(1), 133. Kane, MT (1992). The Assessment of Professional Competence. SAGE Social Science Collections, 163-182. Kift , S, Israel, M & Field, R. 2010. Threshold Learning Outcomes for the LLB. ALTC. Krieger, S. H., & Martinez, S. A. (2012). Performance Isn't Everything: The Importance of Conceptual Competence in Outcome Assessment of Experiential Learning. Clinical Law Review, 19, 251-296. Legal Education and Training Review. (2013). Setting Standards: The future of legal services education and training regulation in England and Wales. Retrieved from http://letr.org.uk/the-report/index.html Maurer, N., & Cole, L. (2012). Design, Teach and Manage: Ensuring Educational Integrity in Field Placement Courses. Clinical Law Review, 19, 115-160. McKinley, R., Fraser, R., Van Der Vleuten, C. & Hastings, A. (2000). Formative assessment of the consultation performance of medical students in the setting of general practice using a modified version of the Leicester Assessment Package. Medical Education. 34:573-579 Mehay, R. (2010). Miller’s Pyramid/Prism of Clinical Competence (1990), http://www.essentialgptrainingbook.com/resources/chapter_10/Millers%2520pyramid%2520 of%2520clinical%2520competence.doc Patrick, C.-J., Peach, D., Pocknee, C., Webb, F., Fletcher, M., & Pretto, G. (2008). The WIL [Work Integrated Learning] Report: A national scoping study. Ram, S. (2008). Industry based learning and variable standards in workplace assessments. Asia Pacific Journal of Cooperative Education 9(2), 129-139. Stuckey, R. T., Barry, M., Dinerstein, R. B., Dubin, J. C., Engler, R., Elson, J. S., .Schwartz, M. H. (2007). Best practices for legal education: A vision and a road map. Columbia, South Carolina. Sullivan, W. M., Colby, A., Wegner, J. W., Bond, L., & Schulman, L. S. (2007). Educating Lawyers: Preparation for the Profession of Law. Sylvester, C. (2015). Problematising Assessment in Clinical Legal Education: Through a glass darkly: Assessment of a real client, compulsory clinic in an undergraduate programme. International Journal of Clinical Legal Education, 23 (1). Tosun, N. & Baris, F. (2011). E-Portfolio Applications in Education . The Online Journal of New Horizons in Education, 1 (4).,42-50. Van Der Vleuten, C. P. M. (1996). The Assessment of Professional Competence: Developments, Research and Practical Implications. Advances in Health Sciences Education, 1, 41-67. Van Der Vleuten, C. P.M., Shatzer, J. & Jones, R., (2001). Assessment of Clinical Competence. Medical Education Quartet in The Lancet, Vol 357, 945-949 Van Der Vleuten, C. P. M., & Schuwirth, L. W. T. (2005). Assessing professional competence: from methods to programmes. Medical Education, 39(3), 309-317. Zegwaard, K., & Coll, R. K. (2003). Assessment of Workplace Learning: A Framework. Asia- Pacific Journal of Cooperative Education, 4(1), 10-18.