UPWARD! HIGHER: HOW A LAW FACULTY STAYS AHEAD OF THE CURVE Cynthia Adams Cynthia Baker Shawn Boyne Max Huffman Andrew Klein Margaret Ryznar Mark Shope Frank Sullivan TABLE OF CONTENTS ABSTRACT ANDREW KLEIN & MAX HUFFMAN, INTRODUCTION MAX HUFFMAN & CYNTHIA ADAMS, ONLINE SIMULATION COURSES Kayleigh Long, Reaction to Huffman and Adams CYNTHIA ANNE BAKER, EXTERNSHIPS: TEACHING, PRACTICE, AND THE BUILDABLE HOUR Jeff Cardella, Reaction to Baker James Strickland, Reaction to Baker SHAWN MARIE BOYNE, GIVING STUDENTS A SEAT AT THE TABLE: USING TEAM-BASED LEARNING IN CRIMINAL LAW Yvonne Dutton, Reaction to Boyne MARGARET RYZNAR, ASSESSING LAW STUDENTS Shawn Marie Boyne, Reaction to Ryznar MARK SHOPE, REAL-WORLD PROBLEM-SOLVING IN THE DIGITAL CLASSROOM Margaret Ryznar, Reaction to Shope FRANK SULLIVAN, JR., TEACHING INTERNATIONAL STUDENTS Tim Walters, Reaction to Sullivan MAX HUFFMAN, SUMMARY AND CONCLUSION http://doi.org/10.18060/4806.1193 414 INDIANA LAW REVIEW [Vol. 51:413 ABSTRACT Full-time and part-time faculty of the IU McKinney School of Law convened together with campus and university partners from the IUPUI Center for Teaching and Learning and Indiana University e-Learning Design and Services for the second annual “Upward!” teaching symposium at the beginning of Fall Semester 2017. The two-day gathering involved panel discussions on topics including online teaching, online course design, teaching externships, designing lessons for the law school’s active learning classrooms, teaching international students, and teaching with an eye to the bar exam. Participants enjoyed a field trip to IUPUI campus offices supporting the university’s teaching mission, including the Center for Teaching and Learning and the recording studio. Panelists contributed to this joint publication, which includes sole- or joint- authored submissions by Professors Adams, Baker, Boyne, Huffman, Ryznar, Shope, and Sullivan; an introduction by Dean Klein and Professor Huffman; and reactions to the primary papers. These submissions reflect a variety of scholarly methods, drawing from empirical study, anecdotal observation, and theoretical analysis. The written submissions from Upward! combine to relate a story of law school curricular, student body, and course design that responds to the changing needs and realities of the profession and student demand. The Introduction by Dean Klein and Professor Huffman explains that design is a combination of top- down direction and bottom-up faculty-led innovation. As law schools continue to cope with uncertainties, including a changing regulatory environment, and resource constraints, the McKinney model offers an approach to purposeful change. 2018] UPWARD! HIGHER 415 INTRODUCTION ANDREW KLEIN* MAX HUFFMAN** Students seek to bust the curve in law school. The law faculty’s job is to ensure that in doing so, students will be well positioned for success in life and the profession. In a constantly changing economy and society, the faculty does its job by keeping ahead of the curve itself. Law today is different from yesterday and the day before, and in any rational world, law teaching must be as well. Change for law schools either happens from within or is forced upon them. The first is preferable, allowing faculties to be innovative and implement change strategically. The second, where it occurs, requires schools to conform to current best practices where others have a first mover advantage. Substantial shifts in student and employer expectations, advances in learning science, economic challenges, and accreditation requirements continue to impact all U.S. law schools. The best approach is to embrace these as opportunities, and this compilation details the work of one faculty in that vein. I. UPWARD! In August 2018, the IU McKinney School of Law will for the third time kick off the academic year with a program by and for faculty, supported by the IUPUI Center for Teaching and Learning and Indiana University e-Learning Design and Services. Upward! began in 2016 as an offshoot of McKinney’s investment in innovative teaching, including significant development of online education. In 2017 Upward! expanded to encompass all parts of McKinney’s teaching mission. Topics included graduate and international programs, externships, experiential learning, bar readiness, student assessment, and online learning—as well as innovative and technology-centric topics such as the school’s active learning classroom and the campus recording studio. McKinney’s Teaching Committee is reprising the project for 2018, with proposed areas of coverage to include online learning, active learning, clinical education, student assessment, capstone courses, and the advanced writing requirement, among other possibilities. Beginning each academic year with a focus on our primary work at McKinney—teaching law students—has enabled * Dean and Paul E. Beam Professor of Law, Indiana University Robert H. McKinney School of Law ** Professor of Law, Director of McKinney Law Online, and Co-Director of Program Assessment and Evaluation. Thank you to our partners in our teaching and learning at Indiana University, particularly the Center for Teaching and Learning and e-Learning Design and Services; to the participants and attendees at the Upward! (Higher) symposium; to the participants in this compilation; and to the editors of the Indiana Law Review for patient and fulsome support of this publication. 416 INDIANA LAW REVIEW [Vol. 51:413 the faculty to direct its energies to continual improvement and adoption of current best practices, helping McKinney students achieve the best educational and professional outcomes. Upward! is one result of a several-years partnership with two campus and university support offices—the IUPUI Center for Teaching and Learning (CTL) and Indiana University’s e-Learning Design and Services (eDS)—through which McKinney pursues continual improvement in teaching and learning.1 As but one example, McKinney’s close connection with these offices has been a primary driver of our online program development. CTL and eDS personnel and leadership are involved in the planning and execution of Upward!, as well as supporting the teaching and scholarly projects that form the core of the seminar. II. THIS COMPILATION Participants in the Upward! program join their voices in this compilation, drawn from the presentations and discussion at the symposium. These written contributions serve as a report on successful efforts and areas for improvement. In each case, the editors of the Indiana Law Review have arranged for a comment (or in some cases two) published immediately after the primary contribution. These comments, from students, alumni, and full-time faculty, give perspectives through a variety of lenses. The compilation begins with the report by Professors Max Huffman and Cynthia Adams2 on a joint project in which they collaborate to produce an online simulation supporting the corporate and commercial curriculum. This collaboration combines Adams’ experience in contract drafting and practice simulations with Huffman’s teaching in the area of comparative competition law. Huffman and Adams reach largely optimistic conclusions about the outcomes and potential for this simulation, while recognizing ample opportunity for future development of the exercise. Professor Cynthia Baker reports on teaching externships,3 with an emphasis on the verb “teaching” as highlighting a key role of the faculty supervisor in ensuring a valuable learning experience. In a well-conceived “externship triangle,” Baker outlines the respective roles of students, faculty supervisors, and placement supervisors, with the student “literally and figuratively, at the top.”4 Baker’s contribution gives credence to the dictum that learning by doing is a 1. See IUPUI Center for Teaching and Learning, https://ctl.iupui.edu [https://perma.cc/ W7SY-5UGY] (last visited May 9, 2018); e-Learning Design and Services (UITS), https://teachingonline.iu.edu/about/eds/index.html [https://perma.cc/T9PT-REM7] (last visited May 9, 2018). The Center for Teaching and Learning has been a long-time partner in the law school’s teaching mission. Our reference here is to the recent push to develop a substantial and sophisticated online presence. e-Learning Design and Services was created to advance Indiana University’s online presence and has been a partner with McKinney Law Online since its inception. 2. See Max Huffman & Cynthia Adams, Online Simulation Courses, infra. 3. See Cynthia Anne Baker, Externships: Teaching, Practice, and the Buildable Hour, infra. 4. See id. 2018] UPWARD! HIGHER 417 misnomer—the learning happens in the process of reflecting on what one has done, and communication and feedback is essential to the learning process. Professor Shawn Boyne advances a related approach to ensuring students control their study in a classroom setting, “[g]iving [s]tudents a [s]eat at the [t]able” through team-based learning.5 Boyne canvases literature showing impressive results from team-based learning in a variety of professional fields and details the results of her own implementation in a first year “bar” course. Boyne’s contribution also highlights the role of university support for faculty innovation, having developed her course while a faculty fellow with the Mosaic Active Learning Initiative.6 In their separate contributions, Professor Margaret Ryznar and Mark Shope each discuss lessons from their work teaching online.7 Ryznar’s study of student attitudes to formative assessments offers one current best practice to ensuring students engage with course material in an online class. Ryznar’s findings are consistent with prior studies of online course design. Shope reports on an online course build for International Business Transactions, one of the core courses in the corporate and commercial and international curricula. Shope’s course requires students to engage with class lessons as if they are law firm associates, preparing them for assignments and interactions with hypothetical supervisors that are representative of ones they can expect to encounter in practice. While not formally a simulation course, Shope uses his personal practice experience to the utmost in this course design. Finally, Professor of Practice Frank Sullivan details his efforts in teaching international students, who are pursuing either LL.M. or J.D. degrees at McKinney.8 Those students join McKinney after completing their primary legal studies at a foreign law school and in many cases after spending some time in law practice. Sullivan’s classes are frequent favorites for those students, who rise to the challenge of U.S. style legal education, with cold calling and oral legal analysis. Sullivan observes the reality of mixed responses to such a full inclusion strategy. However, the grapevine among international students, who encourage each other to take Sullivan’s classes, is ample proof that the students appreciate such deliberate efforts at drawing them in. The following compilation from Upward! offers (1) encouragement, (2) guidance, and (3) careful study regarding the essential process of a law faculty’s keeping itself ahead of the curve. 5. See Shawn Marie Boyne, Giving Students a Seat at the Table: Using Team-Based Learning in Criminal Law, infra. 6. See id. at n.*; see also Mosaic Initiative, MOSAIC: IND. UNIV.’S ACTIVE LEARNING INITIATIVE, https://mosaic.iu.edu/ [https://perma.cc/7HS9-JEYN] (last visited May 10, 2018). 7. See Margaret Ryznar, Assessing Law Students, infra; Mark L. Shope, Real World Problem-solving in the Digital Classroom, infra. 8. See Frank Sullivan, Jr., Teaching International Students, infra. 418 INDIANA LAW REVIEW [Vol. 51:413 ONLINE SIMULATION COURSES MAX HUFFMAN CYNTHIA ADAMS* I. NEED FOR PRACTICAL SKILLS EDUCATION Experiential learning and in particular practice simulations represent a core part of the modern law school curriculum. As we explained in our 2016 Grant Proposal, successfully seeking grant assistance to develop the Contract Drafting Simulation to Accompany Comparative and International Competition Law (CDS Course): Even well-designed, effective, popular courses need regular attention to remain innovative. Combined with new requirements of experiential learning from the accrediting agency, both Contract Drafting and Comparative and International Competition Law would benefit from being revised in a manner that meets the accrediting agency’s requirements for experiential learning. In addition, existing pedagogical techniques for online teaching, while they are proving to be highly effective, are not sufficiently established to produce confidence that there is no room for further innovation.1 We identified one primary innovation—the use of online teaching to address limitations in practical skills training in the law school curriculum.2 With regard to the value of practical skills training in law school, the 2016 Grant Proposal noted: “The target is simulations for several reasons. First, law students * Professor of Law; Clinical Professor of Law. We are thankful for financial support from a Curriculum Enhancement Grant from the IUPUI Center for Teaching and Learning and course design support by Doug Jerolimov, Jeani Young, Michael Jasiak and Michael Anthony. Renee Petrina, Principal Instructional Designer at e-Learning Design Services, joined us in the presentation that gave rise to this submission. 1. Contract Drafting Simulation to Accompany Doctrinal Course, Curriculum Enhancement Grant Proposal (submitted Jan. 2016) (on file with authors) (citing Daniel Thies, Rethinking Legal Education in Hard Times: The Recession, Practical Legal Education, and the New Job Market, 59 J. LEGAL EDUC. 598, 605 (2010) (increased economic pressures on law firms has led to greater demand for law graduates with practical skills and ability to take on complex projects) [hereinafter Grant Proposal]; Robert P. Kuehn, Measuring Clinical Legal Education’s Employment Outcomes, 2015 WIS. L. REV. 645, 661-663 (describing surveys in which legal employers rank practical skills as very important). 2. See Max Huffman, Online Learning Grows Up—And Heads to Law School, 49 IND. L. REV. 57, 62-63 (2015) (summarizing a recent push for greater skills and experiential learning in legal education). 2018] UPWARD! HIGHER 419 increasingly face an employment environment where to get hired they must demonstrate that they have already at least practiced, if not mastered, some of the lawyering skills entailed by the job. Survey research, in fact, supports this claim.”3 Information gathered as part of the law school’s 2016 self-study, through interviews collected in spring 2016, supports the call for greater practical skills education at McKinney. According to an interview report paraphrasing the comments of alumni: In the 1980s the law school seemed to get a bad rap as the “practical” (as opposed to theoretical) law school. From the perspective of a practicing attorney, that is a benefit rather than a liability. It is a pleasure to have new attorneys with practical skills. This is the case in hiring at the associate level as well as the summer law clerk programs.4 One of the most sought-after courses of targeted study among JD students at McKinney is corporate and commercial law (CCL). The CCL program was launched in Academic Year 2013-14 as a concentration and achieved Graduate Certificate status for Academic Year 2014-15.5 The stated purpose of the CCL Graduate Certificate is consistent with the goal of preparing students for practice: Students from the Indiana University Robert H. McKinney School of Law seeking to practice in the competitive fields of corporate and commercial law compete on the job market against top students from law schools across the country. Students that have taken and performed well in a broad selection of corporate and commercial courses perform at or above the expectations of even the most discriminating of employers, rising to the top ranks in their chosen career paths. * * * The Certificate will provide these students a much-needed competitive edge in the job market.6 Opportunities for experiential education in the CCL program have been insufficient to meet student demand. Certificate students are encouraged to pursue experiential credits through offerings that include: • International Commercial Arbitration Moot Court; • Transactional Law Meet competition; • Subject-relevant Moot Court competitions; • Externship placements in corporate counsel offices, Bankruptcy 3. Grant Proposal, supra note 1. 4. Report from the Interview of McKinney Alumni (Spring 2016) (on file with authors). 5. See Graduate Certificate Requirements: Corporate and Commercial Law Graduate Certificate, IND. UNIV. ROBERT H. MCKINNEY SCH. OF LAW, https://mckinneylaw.iu.edu/ courses/certificate_corporate-commercial-law.cfm [https://perma.cc/V9CS-KWMN] [hereinafter Graduate Certificate Requirements] (last visited May 10, 2018). 6. Proposal for a Graduate Certificate in Corporate and Commercial Law (submitted Fall 2015) (on file with authors). 420 INDIANA LAW REVIEW [Vol. 51:413 Courts, or appropriate government agencies; • Simulation courses including Contract Drafting.7 The two of those that involve substantial professor-to-student instruction, the VIS Moot Court and Contract Drafting, are commonly oversubscribed. Thus, we identified a need for additional faculty-taught experiential learning opportunities in the CCL curriculum. II. PARTICULAR NEED FOR ONLINE EXPERIENTIAL LEARNING Distance courses are highly sought-after at McKinney. The courses in the expanding distance catalog, representing approximately 50 credit hours in total, are frequently oversubscribed. Of the courses in the online catalog, prior to this project only one—Contract Drafting8—met accreditor requirements for experiential learning credit. Two of fifty represents four percent of the available distance education credits, substantially less than the ten percent (six of sixty) required by the law school’s accrediting agency for students to take in their two years of upper-level study.9 The project was intended to take one step toward closing that gap by adding two more credits of online experiential learning to the curriculum. Experience in online course development and teaching at McKinney is showing that online teaching techniques lend themselves to experiential education and in particular practice simulations. Online pedagogical techniques have much promise as complements to live experiential offerings. A reality of most of our students’ post-graduate work environments will be the application of legal knowledge and skills through communication at a distance. Likewise, a core design feature of the McKinney online programming—interactive but student driven learning—is a feature of any practice-ready curriculum. One of us has argued that experiential learning may be the most natural pedagogical technique in online law teaching.10 That is because: • Modern team-based law practice frequently involves distance communication; • Supervisor engagement with junior lawyers is based on comments on written work communicated by e-mail, file server, or cloud storage; • Self-education from client files and external research is a common requirement for beginning lawyers that is easily replicated in the 7. See Graduate Certificate Requirements, supra note 5. 8. Several instances of Contract Drafting are taught yearly. Of those, only one instance, taught by Professor Adams for 2 credits, is available online. 9. ABA Standards, AM. BAR ASS’N, https://www.americanbar.org/groups/legal_education/ resources/standards.html [https://perma.cc/HE9Z-C8DJ] (last visited Feb. 9, 2018) (referencing Standard 303). 10. See Huffman, supra note 2, at 64. 2018] UPWARD! HIGHER 421 online environment; • Forms of student-student and instructor-student engagement frequently leveraged in online course design are by their nature representative of law practice.11 The Contract Drafting Simulation is the fullest expression of the effort to introduce practice-ready components to online courses across the McKinney Law Online curriculum, part of the “Phase II Redesign” of online teaching at McKinney. III. CONTRACT DRAFTING SIMULATION ACCOMPANYING A DOCTRINAL COURSE This project aimed to enhance McKinney’s online programming by implementing innovations in existing online courses that have been delivered and are being delivered presently to law students. The project’s target was “practice readiness” through simulated transactional law practice. Law students face an employment environment in which lawyering skills are highly valued at the entry level. The simulation we developed represents an innovation that we expect will be quickly disseminated in the McKinney curriculum as well as externally. The project produced a legal skills “simulation course” running simultaneously and in collaboration with an established doctrinal course, Comparative and International Competition Law. Standard 304 of the ABA Standards for Approval of Law Schools,12 defines simulation courses as follows: (a) A simulation course provides substantial experience not involving an actual client, that (1) is reasonably similar to the experience of a lawyer advising or representing a client or engaging in other lawyering tasks in a set of facts and circumstances devised or adopted by a faculty member, and (2) includes the following: (i) direct supervision of the student’s performance by the faculty member; (ii) opportunities for performance, feedback from a faculty member, and self-evaluation; and (iii) a classroom instructional component.13 The ABA Standards further require that simulations, clinics, and field placements make up six credits, or nearly seven percent, of the JD degree program.14 Because these courses do not count against the thirty-two credit hours of required first- year coursework, these experiential credits represent more than ten percent of McKinney students’ elective coursework. 11. See also infra Part IV. 12. ABA Standards, supra note 9 (referencing Standard 304). 13. Id. 14. Id. (referencing Standard 303). 422 INDIANA LAW REVIEW [Vol. 51:413 A. Backward Design Online courses at McKinney generally, and the Contract Drafting Simulation in particular, are produced through the process of “backward design,” through which designing faculty first identify outcomes sought to be achieved by the course.15 With outcomes in place, designing faculty determine what would constitute evidence of those outcomes and finish the course design by creating learning experiences that will produce that evidence.16 Of this design process, carefully identifying learning outcomes proved to be the most difficult step. One of us had extensive prior experience teaching Contract Drafting as a simulation, both in the live classroom and online. The other had experience in the specific doctrinal subject area covered. Identifying outcomes of a simulation of contract drafting in comparative competition law proved to be a lengthy and iterative process, which we undertook in close consultation with professional course designers employed by the IUPUI Center for Teaching and Learning.17 At the end of the first step in the design process, we had identified the following outcomes for the Contract Drafting Simulation Accompanying Comparative and International Competition Law: At the end of the course, a successful student will: 1. Identify client's objectives and the parties’ mutual interests and gather relevant facts; 2. Research and identify governing legal requirements; 3. Assess risk and evaluate strategic options, arrive at an effective drafting strategy designed to achieve client goals; 4. Draft clear, concise, and precise contract provisions to accomplish client objectives and comply with governing law by: • Critically reviewing and modifying precedent forms • Identifying and resolving ethical issues • Drafting individually from scratch; 5. Engage with colleagues in constructive deliberation, questioning, and critical analysis and plan a strategy to effectively address client goals and concerns; 5. Apply lessons from the Comparative and International Competition Law course in practical exercises simulating real world lawyering. Of the six (or eight, including sub-parts) outcomes, a substantial percentage are at the level of “synthesis and evaluation” on the leading Bloom’s Taxonomy 15. See GRANT WIGGINS & JAY MCTIGHE, UNDERSTANDING BY DESIGN 13-35 (2d ed. 2005). 16. Id.; see, e.g., Huffman, supra note 2, at 73-75 (describing backward design as a process of “bespoke design” employed at McKinney). 17. See generally IUPUI CENTER FOR TEACHING AND LEARNING, https://ctl.iupui.edu/ [https://perma.cc/YRE7-A7SD] (last visited May 10, 2018). 2018] UPWARD! HIGHER 423 of Educational Objectives.18 Research and experience in legal education demonstrates that such synthesis and evaluation is essential to preparing students for success on the bar exam and in practice.19 Step Two in the backward design process was to identify evidence of success in achieving the course outcomes. Our combined experience in the doctrinal subject and transactional practice and pedagogy proved an effective background for this step in the process.20 These evidentiary products fell into four primary categories: • Drafting increasingly complex contract provisions; • Researching and applying relevant legal rules; • Communicating with and advising hypothetical clients with regard to the representation; • Reflecting on the exercises in structured fashion. As design progressed, we became convinced of the importance of the structured reflection as a complement to the particular learning experience.21 Research suggests that “experiential learning” is achieved through the process of reflection rather than merely through experience.2 2 Di Stefano, Gino, Pisano and Staats produced experimental results demonstrating that experience combined with reflection is a more effective learning process than experience alone.2 3 As a result, fully one third of the assignments in the simulation required students to reflect on the outcomes of other work. In Step Three we created learning experiences producing the evidence (of writing, of research, of client counseling and communication, and of reflection) that we had previously identified as necessary to gauge success in meeting the learning outcomes. These learning experiences were intended to combine hard- hitting legal work with the kind of intellectual engagement that is expected in a complex transactional practice. We thus built increasingly complex drafting 18. See generally BENJAMIN S. BLOOM ET AL., TAXONOMY OF EDUCATIONAL OBJECTIVES: THE CLASSIFICATION OF EDUCATIONAL GOALS (1956). 19. Cf. Nicholas L. Georgakopoulos, Bar Passage: GPA and LSAT, Not Bar Reviews 21-22 (IU McKinney Sch. of Law, Research Paper No. 2013-30, 2013), available at https://papers.ssrn. com/sol3/papers.cfm?abstract-id=2308341 [https://perma.cc/HKV2-UMPG]. 20. See, e.g., CYNTHIA M. ADAMS & PETER K. CRAMER, DRAFTING CONTRACTS IN LEGAL ENGLISH: CROSS-BORDER AGREEMENTS GOVERNED BY U.S. LAW (2013); see also Huffman, supra note 2 (discussing experience teaching Comparative and International Competition Law); Max Huffman, Teaching Antitrust Online, June 2015 CPI Antitrust Chron., at 6. 21. Cf. Susan M. Land, Supporting Reflection in Online Learning Environments, in EDUCATIONAL MEDIA & TECHNOLOGY YEARBOOK 141-54 (2009) (reviewing strategies for student reflection in online learning). 22. See, e.g., Giada Di Stefano et al., Making Experience Count: The Role of Reflection in Individual Learning, (Harv. Bus. Sch., Working Paper No. 14-093, 2016), available at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2414478 [https://perma.cc/B8ZV-KGLR]. 23. Id. at 4. Particular thanks to Doug Jerolimov for advancing this argument. 424 INDIANA LAW REVIEW [Vol. 51:413 assignments; discussion boards related to students’ work; a “mind-mapping” exercise requiring students’ to demonstrate awareness of the interrelatedness of contract provisions; and client interview and communication assignments that served as building blocks to the more complex assignments while also supporting their own learning outcomes. B. Foundational Materials and “Scaffolding” Assignments can be “scaffolded” such that early assignments serve as building blocks for later assignments. While scaffolding is believed to be universal in course design—later lessons in a course commonly build on earlier lessons—scaffolding is a fundamental design principle in online course design and in experiential learning in particular.24 Properly designed, scaffolded exercises require learners to reach “just above the learner’s current skills level” with each new assignment, helping the learner to “reach new levels of learning.”25 Thus, scaffolded exercises in the contract-drafting sphere might begin with drafting a small and easily separable contract provision, move to analyzing the interrelationships of contract provisions, and finish with the students’ drafting a complete contract. We employed scaffolding in the learning experiences in the simulation, leading from basic assignments to a single, highly complex piece of transactional drafting, in which students produced a “breakup fee” provision in a hypothetical cross-border joint venture agreement. Rather than drafting a full contract, students finished their work with a complex and highly integrated provision, the content of which drew on earlier drafting assignments as well as research and client advice experience students gained interstitially. In their final form, the simulation’s scaffolded assignments arranged themselves this way: • Simple drafting exercise (confidentiality clause) • Simple drafting exercise (purpose statement) • Holistic contract appraisal exercise (mind-mapping the constituent parts of the contract) • Counseling exercise (client advice letter) • Complex drafting exercise (breakup fee provision) The point allocation for grading purposes followed the pattern of lesser to increased importance of the class assignments as the assignments became increasingly complex and relied on past learning. 24. See Rose M. Baker, Examples of Scaffolding and Chunking in Online and Blended Learning Environments, (Penn State, Short Practitioner Paper, 2010), available at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1608133 [https://perma.cc/4JYU-UQPQ]. 25. Id. at 3. 2018] UPWARD! HIGHER 425 IV. EVALUATING THE PROJECT The law school offered the online CDS Course as a one-credit adjunct to the three-credit doctrinal course in Summer 2017. The summer term is on a compressed schedule, permitting a total of six weeks of instruction for the one- credit course (and nine weeks for the corresponding three-credit doctrinal course). This compressed schedule requires a higher workload per week than is usual for a full-semester course: instead of three hours weekly (one in class, two in preparation), students were expected to devote approximately six hours weekly (two “in class,” four in preparation) to earn their one course credit. A. Student Retention After an initial enrollment of nearly twenty students in the CDS Course, eleven saw the simulation through to completion. Most, if not all, of the drops from the course occurred before the course’s commencement or during the first week of the course. Retention rate is a frequent concern with online education generally, with recent scholarship noting retention problems counterbalancing increased interest in online courses.26 As a general matter, McKinney has not encountered retention issues in its online courses, making the school’s experience perhaps unique relative to online education generally. The fifty-five percent retention rate in the CDS Course is almost certainly explainable by three things, which combined to produce a unique workload problem. First, we offered the course in the compressed summer term, allowing students six weeks to cover a substantial amount of material. Second, we offered the course together with a three-credit doctrinal course, also taught in a compressed schedule. Third, careful reflection on the course build has convinced us that the assignments were too many and too substantial for the single course credit offered, even under normal circumstances. B. Course Content The CDS Course successfully provided a simultaneous “transfer of learning” opportunity, bringing together legal theory and skills to create a unique, complex role-playing experience for students. Although the course material and assignments were challenging, the students were exceptionally motivated and highly engaged in all aspects of the course. Based on students’ strong performance in the CDS Course, the authors have concluded that (1) the videotaped lectures, PowerPoints, and supporting reading materials provided suitable overviews and introductions to targeted lawyering skills27 and (2) the assignments afforded students appropriate opportunities to 26. See, e.g., Papia Bawa, Retention in Online Courses: Exploring Issues and Solutions—A Literature Review, SAGE OPEN 1-11 (Jan.-Mar. 2016). 27. The simulation’s first module, a primer covering fundamental drafting principles, prepared students for the upcoming drafting assignments. Even so, students who had completed 426 INDIANA LAW REVIEW [Vol. 51:413 apply and hone those targeted skills. Further, students appeared satisfied with the course on the basis of anonymous student evaluations of teaching. C. Areas for Improvement Nevertheless, despite the success of the CDS Course’s pedagogical construct, the authors believe there is room for improvement. When designing a collaborative simulation course, professors must give careful attention to (1) synchronizing topics covered in the simulation course and doctrinal course, (2) the number and complexity of simulation assignments, (3) the length of time allotted to teach the simulation course, and (4) the credit hours earned. These factors are intimately interrelated, each influencing the others. When designing the CDS Course, the authors continually reassessed their course plan in light of these factors. But experience with the maiden run of the CDS Course indicates that they could have been even more conservative in their assessment. In a practice simulation course, students require repeated opportunities to hone skills, and because each assignment builds on previous assignments, ample time to write, revise, and reflect on their work product. In turn, the professors teaching the simulation course require sufficient time to provide detailed feedback to each student so that the student can assimilate this information and implement improvements when completing the next assignment. Consequently, adequate time must be built into the syllabus to allow for feedback and student assimilation. For a skills course taught in a traditional fall or spring semester, assignments may be spread over thirteen weeks. But determining the appropriate number and pacing of assignments in a skills course running in a greatly condensed time frame is challenging, as the authors discovered when offering the one-credit hour CDS Course during a six-week period in the summer. The course’s short duration significantly constricted the timing and turnaround of assignments. Some students in the CDS Course commented that they occasionally felt overwhelmed by the workload. Therefore, before offering the course again, the authors will be reassessing the assignments to determine whether assignments can be trimmed to fit more comfortably into a condensed term. Basic Contract Drafting, a standalone drafting course, prior to enrolling in the CDS Course understandably had a deeper understanding and mastery of drafting principles and were more adept at applying these principles to the drafting assignments, as compared to students who did not have the benefit of the foundation course. Accordingly, the CDS Course as presently constructed should not be considered a replacement for a course focused entirely on teaching drafting principles. 2018] UPWARD! HIGHER 427 REACTION TO HUFFMAN AND ADAMS, ONLINE SIMULATION COURSES KAYLEIGH LONG* While employment rates are on the rise among recent law school graduates, only “73 percent of the 2016 graduates of the 204 law schools approved by the ABA . . . were employed in full-time long-term Bar Passage Required or J.D. Advantage jobs roughly 10 months after graduation.”1 One critical way for law students to join the ranks of the three out of four who are employed is to gain practical skills that can be immediately deployed in their future work environments. While law students have typically picked up these skills through clinics, externships, and internships, students with packed daytime schedules and working students attending evening classes may not be available to take advantage of these opportunities. To meet the needs of all law students, law schools should consider adding more online simulation courses to their curriculum. Before launching an online simulation course, law schools and the faculty running the courses must plan extensively the objectives, design, and feedback of the course. Huffman and Adams rely on the concept of backward design, employed by teachers in all levels of education, to think first about what law students will have learned and practiced by the end of the course and then working backwards to the first day of class in creating and designing content and assignments. Further, as Huffman and Adams explain, the assignments selected should be “scaffolded” to build upon student learning while at the same time providing professors with instructive feedback on topics that students have mastered and topics that may need to be revisited. Ideally, students will be able to use one or more of these assignments as writing samples to show potential employers, outside of their transcript, the practical skills they gained from the course. As law schools look to expand their online course offerings, Huffman and Adams remind law school administrators and faculty that the design, planning, and execution of these courses, especially simulation courses, cannot be taken lightly or done haphazardly. When done effectively, the short and long term benefits of such an experience can be enormous for a law student seeking to stand out in the applicant pool. * J.D. Candidate, 2018, Indiana University Robert H. McKinney School of Law— Indianapolis, Indiana; M.A. in Teaching, 2014, Oakland University—Rochester, Michigan; B.A. 2012, Indiana University—Bloomington, Indiana. 1. ABA legal education section releases employment data for graduating law class of 2016, AM. BAR ASS’N (May 11, 2017), https://www.americanbar.org/news/abanews/aba-news-archives/ 2017/05/aba_legal_education.html [https://perma.cc/KK4D-CMKS]. 428 INDIANA LAW REVIEW [Vol. 51:413 EXTERNSHIPS: TEACHING, PRACTICE, AND THE BUILDABLE HOUR CYNTHIA ANNE BAKER* Externships, traditionally the outer boundary of clinical pedagogy, are becoming central to how law schools are providing practical experience in conjunction with supervised reflection on the practice of law. As pressures exert change, externship faculty should be mindful about how externships meet the needs of today’s law students. Ultimately, externships affect the entire fabric of the legal community: students, supervising lawyers, faculty, and the legal profession as a whole.1 Eight years and 227 externship law students later, I still whole-heartedly agree with sentences one and three. However, if I were to re-write the second today, I would change it to, “[a]s pressures exert change, externship faculty and supervising lawyers should be mindful . . . .” Since I co-authored Under Pressure, the economy’s recovery, vis-a-vis the legal profession, continues,2 but a commensurate depressurization of teaching externships has not. Rather, the pressures of offering and teaching high quality externships at our nation’s law schools have intensified. From my vantage point, this increased pressure arises from three primary sources: continuing evolution of philosophies regarding the educational value of externships,3 increasing costs of attending law school,4 and a wide-spread “desire to improve the competence of novice lawyers.”5 * Clinical Professor of Law, Director of Experiential Learning, and Director of the Program on Law and State Government, Indiana University Robert H. McKinney School of Law. Thanks to Andrew Klein and Joel Schumm for their participation in the panel presentation that gave rise to this submission. 1. Cynthia A. Baker & Robert Lancaster, Under Pressure: Rethinking Externships in a Bleak Economy, 17 CLIN. L. REV. 71, 98 (2010). 2. See generally GEORGETOWN LAW CENTER FOR THE STUDY OF THE LEGAL PROFESSION, REPORT ON THE STATE OF THE LEGAL MARKET (Thomson Reuters 2017), available at http://legalsolutions.thomsonreuters.com/law-products/solutions/peer-monitor/complimentary- reports [https://perma.cc/9RBW-6Y3J]. 3. For a valuable history of the evolution of legal externships and scholarship on teaching externships, see Harriet N. Katz, The Past and Future of Externship Scholarship, 23 CLINICAL L. REV. 397, 399-403 (2016). 4. See generally U.S. Gov’t Accountability Office, Higher Education: Issues Related to Law School Cost and Access, GAO 10-20 (Oct. 26, 2009) (reporting that primary reasons for increased costs of law school include strategies by law schools to offer more hands-on, resource intensive approach to legal education and competition among law schools for higher rankings. For public law schools, another primary driver of increased tuition costs also includes decreases in state funding). 5. Katz, supra note 3, at 421. 2018] UPWARD! HIGHER 429 In response to these pressures, the American Bar Association (ABA), the accrediting body for this nation’s law schools,6 has recently changed its standards regarding externships. In 2015, the ABA, for the first time in its history as an accreditor, set a required minimum of six credit hours of an experiential learning for every law school graduate.7 With this new standard, externships continued their trajectory from the “hidden curriculum,” to an effort recognized by the Association of American Law Schools, to an important aspect of an ABA requirement.8 In 2016, the ABA combined former ABA Standard 305 (“field placements and other study outside the classroom”) into revised Standard 304 (“simulation courses, law clinics, and field placements”) and, in so doing, provided more guidance (some would say micromanagement) regarding the nature and structure of externships offered at our nation’s law schools.9 At the same time, the ABA also negated the long-standing interpretation for former Standard 305 that prohibited law students from earning academic credit for “participation in a field placement program for which the law student receives compensation.”10 Together, these developments have invited me to rethink how I teach externships and, in particular, how I can more effectively create environments of shared pedagogy. Ideally, efforts toward increased collaboration and 6. See Accreditation in the United States: Specialized Accrediting Agencies, U.S. DEPT. OF EDUC., https://www2.ed.gov/admins/finaid/accred/accreditation_pg7.html#NationalInstitutional [https://perma.cc/M4MR-UJMV] (last visited May 10, 2018). 7. SECTION OF LEGAL EDUCATION AND ADMISSIONS TO THE BAR, AM. BAR ASS’N STANDARDS AND RULES OF PROCEDURE FOR APPROVAL OF LAW SCHOOLS, 2015-16, Standard 303(a)(3), at 16-17 (2015). Regarding experiential learning, the Standard reads: A law school shall offer a curriculum that requires each student to satisfactorily complete at least the following: . . . (3) one or more experiential course(s) totaling at least six credit hours. An experiential course must be a simulation course, a law clinic, or a field placement. To satisfy this requirement, a course must be primarily experiential in nature and must: (i) integrate doctrine, theory, skills, and legal ethics, and engage students in performance of one or more of the professional skills defined in Standard 302; (ii) develop the concepts underlying the professional skills being taught; (iii) provide multiple opportunities for performance; and (iv) provide opportunities for self- evaluation. 8. Katz, supra note 3, at 399-400 (citing Marc Stickgold, Exploring the Invisible Curriculum: Clinical Field Work in American Law Schools, 19 N.M. L. REV. 287 (1989)). 9. SECTION OF LEGAL EDUCATION AND ADMISSIONS TO THE BAR, AM. BAR ASS’N STANDARDS AND RULES OF PROCEDURE FOR APPROVAL OF LAW SCHOOLS, 2016-17, Standards 303(a)(3), 304(c), at 16-18 (2016). [hereinafter 2016 STANDARDS AND RULES] 10. One of the Interpretations to former ABA Standard 305 read, “A law school may not grant credit to a [law] student for participation in a field placement program for which the student receives compensation. This Interpretation does not preclude reimbursement of out-of-pocket expenses related to the field placement.” SECTION OF LEGAL EDUCATION AND ADMISSIONS TO THE BAR, AM. BAR ASS’N STANDARDS AND RULES OF PROCEDURE FOR APPROVAL OF LAW SCHOOLS, 2014-15, Interpretation 305-2, at 19 (2014). 430 INDIANA LAW REVIEW [Vol. 51:413 communication between and among the three primary externship participants (the student, the supervising lawyer, and the faculty advisor) can intensify the focus on the externship student’s learning and professional progress throughout the externship semester. Now, more than ever before, externship faculty and supervising attorneys have reason to bring teaching and practice together for the benefit of the externship student. Well-designed externships have the potential of encouraging all of the externship participants to practice the habit of buildable hours, hours that contextualize professionalism and reflection as important aspects of the practice of law. Part I of this contribution offers a simple visualization for legal externships, one which, at its essence, promotes a holistic and realistic approach to teaching externships to today’s law students. After addressing some awkward terminology associated with legal externships, Part I encourages treating each communication line of the “externship triangle,” as additional “experiential texts” from which each externship participant can teach and learn. Part II asks the question, “but what should happen in an externship?” and explores three specific design ideas supported by ABA Standard 304(c): a) recognizing the symbiotic nature of the faculty advisor’s responsibility to evaluate each student’s “educational achievement” and the supervising lawyer’s responsibility to “provide a substantial lawyering experience,” b) incorporating the externship student’s individualized educational objectives into the required “written understanding,” and, c) expanding on the feedback and self-evaluation requirement to include all three participants, not just the law student extern. This contribution concludes that by more fully engaging all of the participants in externship teaching, externships can be a lynchpin to stronger, healthier profession. I. THE EXTERNSHIP TRIANGLE, ITS CONTENT, AND ITS EDGES The externship triangle (graphical depiction in Figure 1, below) conveys that a student’s legal externship comprises, first and foremost, the student’s educational objectives and what the student is learning and doing to reach those throughout the externship semester. Its vertices represent the three participants in the externship: the student, the faculty advisor, and the supervising lawyer. Its sides represent that all three of these people need to be communicating and, indirectly, what those three people should be communicating about: what the student is learning and doing for the academic externship, the student’s educational objectives (and their evolution throughout the externship semester), the type of feedback, guidance, and teaching the student is receiving from his or her supervising lawyer and his or her faculty advisor. 2018] UPWARD! HIGHER 431 Figure 1. Adult learning theory (andragogy) has as its “central tenet . . . that adult learners expect a mutuality of control and respect between teacher and student.”11 Well-designed externships reinforce this basic tenet to good effect. While scholars disagree about the emphasis of the relative importance of each participant’s respective role in an effective externship,12 the externship triangle makes clear that the student is, literally and figuratively, at the top. The basic responsibilities for each person involved in legal externships are set by ABA Standard 304(c).13 The externship student’s responsibilities include performing legal work; the supervising lawyer is primarily responsible for assigning properly gauged legal work to the externship student throughout the semester; externship faculty are responsible for designing quality learning and communication environments (through seminar courses, online portals, conferences, phone conversations, individual tutorials, guided reflections, or a combination of some or all of these) to facilitate and complement the substantive goals of the externship, including evaluation of each student’s “educational achievement.” Beyond those individual participant assignments of 304(c), I suggest that the lines of communication between and among all three of the people engaged in each externship are essential to helping the externship law student learn more deeply about what he or she is experiencing and observing in the externship 11. See Elizabeth G. Ford, Toward a Clinical Pedagogy of Externship, 22 CLINICAL L. REV. 113, 118 (2015) (citing Frank Bloch, The Andragogical Basis of Clinical Education, 34 VAND. L. REV. 321, 330 (1982)). 12. See id. at 119-21 (citing competing pedagogical theories). 13. See infra note 15 (quoting the Standard). 432 INDIANA LAW REVIEW [Vol. 51:413 setting. The variety of externships offered at our nation’s law schools touch on a wide scope of substantive law comprising the work of lawyers and, perhaps, an even wider scope of pedagogical approaches to their teaching.14 However, regardless of the subject matter and approaches of the points of the “externship triangle,” the lines of the “externship triangle” are just as important as its points. With the entrance of the reformulated ABA Standard 304(c), new parameters have been established for externships and their teaching, but some awkward terminology remains.15 While the Standard references a “field placement course,” 14. See, e.g., ROBERT R. KEUHN & DAVID A. SANTACROCE, THE 2016-17 SURVEY OF APPLIED LEGAL EDUCATION, at 10-11 & Table 7 (CSALE 2017), available at http://www.csale.org/files/ Report_on_2016-17_CSALE_Survey.pdf [https://perma.cc/QC7L-TMQU] [hereinafter 2017 CSALE Report]. 15. 2016 STANDARDS AND RULES, supra note 9, at 16. ABA Standard 304(c): (c) A field placement course provides substantial lawyering experience that (1) is reasonably similar to the experience of a lawyer advising or representing a client or engaging in other lawyering tasks in a setting outside a law clinic under the supervision of a licensed attorney or an individual otherwise qualified to supervise, and (2) includes the following: (i) direct supervision of the student’s performance by a faculty member or site supervisor; (ii) opportunities for performance, feedback from either a faculty member or a site supervisor, and self-evaluation; (iii) a written understanding among the student, faculty member, and a person in authority at the field placement that describes both (A) the substantial lawyering experience and opportunities for performance, feedback and self- evaluation; and (B) the respective roles of faculty and any site supervisor in supervising the student and in assuring the educational quality of the experience for the student, including a clearly articulated method of evaluating the student’s academic performance; (iv) a method for selecting, training, evaluating and communicating with site supervisors, including regular contact between the faculty and site supervisors through in-person visits or other methods of communication that will assure the quality of the student educational experience. When appropriate, a school may use faculty members from other law schools to supervise or assist in the supervision or review of a field placement program; (v) a classroom instructional component, regularly scheduled tutorials, or other means of ongoing, contemporaneous, faculty-guided reflection; and (vi) evaluation of each student’s educational achievement by a faculty member; and (vii) sufficient control of the student experience to ensure that the requirements of the Standard are met. The law school must maintain records to document the steps taken to ensure compliance with the Standard, which shall include, but is not necessarily limited to, the written understandings described in Standard 304(c)(iii). Id. 2018] UPWARD! HIGHER 433 Standard 304(c) addresses externships by definition and design.16 That is, while externships are referenced as “field placements” in the ABA Standards, the term externship is widely employed in academic literature,17 professional conferences,18 and law schools.19 In legal academia, internships differ from externships primarily in their disconnection from academic credit. Like they would in externships, law students involved in legal internships are working with lawyers in a legal setting and, presumably, are doing legal work. However, because no academic credit is awarded, internships do not necessarily involve faculty engagement with the law student’s experience nor any formal assessment of the student’s work or educational progress. In contrast, the legal externship aims to connect the substantive law and skills that the students are learning at their respective externships with support from the law school. As a teacher, I have long seen the students’ individual and collective externship experiences as the “texts” from which each student, or group of externship students, learn.20 More recently, I have come to appreciate the importance of the lines of communication between and among the externship participants as equally important texts. This change in perception has prompted some exploration and experimentation about how I can better communicate the “inside of the law school” aspects of the externship with the respective externship’s supervising attorney. II. TEACHING AND DESIGN IDEAS THAT BUILD TOWARD BUILDABLE HOURS Assuming a basic structure for legal externships, what should happen during the 50 - 250 hours that a law student devotes to this educational endeavor?21 Is the 16. See id. 17. See, e.g., Katz, supra note 3 (employing term “externship” to refer to field placements for credit in legal education). 18. Externships 9: Coming of Age, held in Athens, Georgia in March 2018 was the most recent iteration of the preeminent conference series for legal externship faculty. Held biannually since 2000, the Externship conference series has brought together externship faculty and others engaged in this area of law school teaching. See Externships 9: Coming of Age, UNIV. GA SCH. L., available at http://www.law.uga.edu/x9 [https://perma.cc/DEQ5-2EWF] (employing term “externship” to refer to field placements for credit in legal education) (last visited Apr. 19, 2018). 19. See, e.g., 2017 CSALE Report, supra note 14 (employing term “externship” to refer to field placements for credit in legal education). Compare Rebecca B. Rosenfeld, The Examined Externship is Worth Doing: Critical Self-Reflection and Externship Pedagogy, 21 CLINICAL L. REV. 127, 128 (2014) (offering a nuanced typography of field placements, externships, practicum classes, and hybrid clinic-externships). 20. See Ford, supra note 11, at 121 (“Even in the midst of this fragmentation, there is some emerging consensus surrounding a few core principles. First, the design of an externship course must place the live experience at the center, whether that is as ‘text’ for the course or as the site of learning.”). 21. See 2017 CSALE Report, supra note 14, at 34-35 (reporting that the most common hours 434 INDIANA LAW REVIEW [Vol. 51:413 faculty advisor teaching the externship or merely monitoring the externship? Is the supervising attorney charged with merely assigning quality legal projects or is he or she responsible for the student’s growth and learning throughout the semester? Is the externship student supposed to be learning substantive law, skills, professional responsibility, professionalism, or all of the above? Traditionally, the answers to all of these questions would rest with the faculty advisor to the externship. However, as law school faculty and law schools deal with the pressures and changes with which this contribution begins, I suggest that the answers to these questions might be better met with more collaboration between and among externship faculty and supervising lawyers. While Standard 304(c) outlines the basic responsibilities for those involved in the externship, the standard invites new ways to build on those basics to enhance the externship student’s opportunity to contextualize professionalism and reflection as important aspects of the practice of law. That is, building on the basic responsibilities set forth in Standard 304(c), more communication about teaching styles, educational objectives, and feedback could lead to buildable hours, and not just for the law student extern, for the faculty advisor and the supervising attorney as well. A. Symbiotic Relationship between the Responsibilities of the Supervising Attorney and the Faculty Advisor Due to the symbiotic nature of the assigned responsibilities under Standard 304(c), externship faculty might consider possibilities around creating an environment of shared pedagogy: sharing the externship course syllabus with supervising lawyers, inviting a cohort of supervising lawyers and judges to be a part of a class discussion (either in person or on-line) regarding a shared text, or creating an environment inviting all involved in the externship (rather than just the law student) to challenge themselves to identifying an educational objective in conjunction with the externship at hand. Similarly, the supervising lawyer could be more involved by comparing notes with the faculty advisor about how he or she carries out assessment responsibilities in the externship work environment, sharing how he or she periodically assesses his or her own professional progress, or by contributing to a faculty advisor/supervising attorney book club (or article club) to meet, without students present, to discuss a current legal topic or an issue related to externship teaching. B. Written Understanding Might Include Student’s Individual Educational Objectives The requirements of the written understanding as set forth in Standard 304 go a distance toward establishing the quality and scope of the externship.22 Including the externship student’s individual educational objectives into that a student must work during the semester for one academic credit were 50 hours, but noting a range of externship hours per academic credit from 45 hours to 60 hours). 22. See 2016 STANDARDS AND RULES, supra note 9, at 16. 2018] UPWARD! HIGHER 435 written understanding would invite conversations and correspondence about those objectives earlier in the semester (or, in some cases, even before the semester begins). Moreover, involving the supervising lawyer and faculty advisor to focus, even if just briefly, on the student’s individual educational objectives would seem to have the ineluctable result of actually improving the quality of the externship, just by virtue of a shared understanding of what, exactly, the student wants to learn. The administrative burden of this approach, requiring the externship participants to draft, review, revise, and even negotiate an individualized agreement for every externship, even just including something more than the boilerplate language required by the ABA, may seem daunting. But by putting the students in charge of the individualized agreements, the students are immediately placed in a professional role with respect to legal education, are drafting their educational objectives in a way that can demonstrate (or not) their understanding of the substantive legal work of their externship, and are communicating with the supervising lawyer and the faculty advisor for purposes of completing a professional, collaborative task. When done well, this approach to the written understanding required by 304(c)(iii) beautifully demonstrates Professor Elizabeth Ford’s premise that “[e]xternship courses, in which students have one foot in and one foot out of the classroom, are perfectly suited to help them rediscover their own agency.”23 For those interested in this approach to the required written understanding, I suggest a phased approach to drafting process. For example, the student could, first, submit three specific educational objectives for review by the faculty advisor during the first week of the externship semester. Then, after feedback from the faculty advisor of those objectives (with consideration of the student’s prior legal and professional experience and knowledge of the legal work afforded by the externship), the student could begin work drafting the final externship agreement, to discuss and review with his or her supervising attorney. After review of the entire agreement, including the individualized educational objectives, by the supervising attorney, the student would be responsible to circulate the final document for signature of all involved.24 This approach to the required written understanding has great potential to increase both the mutual 23. Ford, supra note 11, at 133. 24. I began implementing this approach to externship agreements in 2012. The template language for externship agreements in my course reads as follows: The educational objectives of [this] externship course include offering externship students the opportunity to . . . set and meet the student’s individualized educational objectives, as outlined here, and as further developed with an individualized development plan (IDP) as the semester progresses. I, [student name] seek to learn specifically during this externship semester, the following: i. ii. iii. . . . . Cynthia A. Baker, Externship Agreement Template (on file with author). 436 INDIANA LAW REVIEW [Vol. 51:413 understanding of the scope of legal work and opportunities available through the externship, as well as and to put the externship student in charge of his or her educational experience in a way that a syllabus, with faculty drafted educational objectives, handed “down” from the faculty advisor, simply does not invite. C. Expanding on the Feedback and Self-Evaluation Requirement Under the new Standards, opportunities exist for all involved in the externship to think a little bit more about the feedback and self-evaluation that are a part of every high-quality externship. That is, based on ABA Standard 304(c)(ii), the faculty advisor and the supervising attorney have an opportunity to facilitate a conversation or share information about feedback and self- evaluation. Given that an understanding about how either or both of these externship participants plan to carry this out for the benefit of the externship student would seem necessary, why not explore the potential that such an exchange might have to inure to the benefit of all three participants in the externship endeavor? One of my favorite guides on this topic is Douglas Stone and Shelia Heen, Thanks for the Feedback.25 Although the book is chock full of approaches and strategies to giving, receiving and assessing feedback, the heart of it can be found in three practices, paraphrased and summarized for purposes of a law school externship teaching, here: 1. When giving feedback, consider what words or messages are going to help the student do a better job next time and to become a better lawyer. What words or messages from the supervising lawyer or faculty member will recognize what the law student did well and point out areas for reconsideration or improvement? 2. Whether the giver or receiver of the feedback, be conscientious about the separation of the judgment of the evaluation (e.g., Great! Awful! Confusing! Late!) from assessment and consequences of the feedback. All parties need to recognize that doing this well takes time and thought and practice and, sometimes, tact and diplomacy. 3. All involved should might consider giving ourselves scores on how we handled (giving or receiving) the feedback. As a student, how did I respond to the feedback? As the supervising lawyer, what was the quality of the feedback I gave? As the faculty advisor, did the feedback the student received prompt any self-reflection, consequences, evolution in educational objectives?26 25. DOUGLAS STONE & SHEILA HEEN, THANKS FOR THE FEEDBACK: THE SCIENCE AND ART OF RECEIVING FEEDBACK WELL (EVEN WHEN IT IS OFF BASE, UNFAIR, POORLY DELIVERED, AND FRANKLY, YOU’RE NOT IN THE MOOD) (2014). 26. Id. at 205. 2018] UPWARD! HIGHER 437 These three examples demonstrate how creating an environment involving the supervising lawyers to externships in the “inside the law school walls” aspects of an externship has great potential to enhance the important lines of communication between the supervising lawyer and the faculty advisor. By increasing the subjects of such communication to include recognizing the symbiotic nature of the responsibilities of the faculty advisor and the supervising attorney, both would more likely focus on what the student is learning (rather than merely what the student is doing). Engaging in purposeful discussion and drafting of the student’s individualized educational objectives for a particular semester allows the student to take charge of the educational goals and may have the ancillary benefit of encouraging the supervising lawyer and the faculty advisor to take stock of their own. Being mindful that feedback for the externship student’s benefit, too, has the potential to become a three-way street that might improve not only the student’s externship experience but the experience of giving and receiving feedback for all involved. If the implementation of any one of these strategies, in even the smallest way, prevents an externship experience from going on auto- pilot, its implementation will have been a success. To the extent an externship infrastructure can contribute to the existence of more buildable hours, I think it is worth building. But the building needs to be done by all three participants to the externship: the student, the supervising lawyer, and the faculty advisor. CONCLUSION As law students move from law school toward their syllabus-free lives as lawyers, externships are not just an introduction to professionalism, but a foundation for professional life that integrates practicing substantive law, self- directed teaching, and the concept of buildable hours. Taught well and experienced fully, externships can be a linchpin to more satisfied members and leaders in the legal profession. With guidance of prepared and engaged externship faculty and supervising lawyers, externships should provide an excellent opportunity for law students to contextualize professionalism and reflection as important aspects of the practice of law. For best effect, the externship student, the supervising lawyer, and the faculty advisor need to be inextricably engaged in this important aspect of our shared profession. 438 INDIANA LAW REVIEW [Vol. 51:413 REACTION TO BAKER, EXTERNSHIPS: TEACHING, PRACTICE, AND THE BUILDABLE HOUR JEFF CARDELLA* As an adjunct professor and solo practitioner, I often feel that I have one foot in the courtroom and one foot in the classroom. I am well aware of the difference between these two worlds and the importance that externships can have in helping students transition from their studies into the practice of law. I have recently served as a placement supervisor for an extern, which I found to be a rewarding experience. Externships allow for a great deal of flexibility, as they lack the rigid structure of the classroom. However, this lack of structure can also present unique challenges. Professor Baker has been involved in well over 200 externships and as such the advice she offers in her seminar and article, Externships: Teaching, Learning, Practice and the Buildable Hour provide a great deal of insight into avoiding common “rookie” mistakes. Professor Baker begins by distinguishing between externships and internships. The specific ABA requirements for externships are explained, as well as the flexibility within these standards and room to tailor the externship around the student—depending on the specific externship. She also addresses the need for a “goals and expectations memo” to be agreed to by the student, the professor and the supervising attorney. Additionally, the use of time logs, hourly recording, obtaining feedback from the supervising attorney, and communications between the professor and supervising attorney—including how the student’s work will be assessed—are discussed. Professor Baker also discusses potential confidentiality issues that can arise, such as when a student externs with a judge and the supervising faculty member has cases pending in front of that same judge. The seminar ends by covering the law school’s educational objectives, the importance of having students set their own individual goals, and ensuring that those goals are met. Professor Baker’s contribution to this symposium addresses many of the same issues, but also outlines why externships are receiving greater emphasis from the ABA, including falling bar passage rates and the demand for “practice-ready” lawyers. Serving as a supervising attorney is a rewarding experience and (for those who do not wish to reinvent the wheel) Professor Baker’s contribution provides a great deal of guidance to those who wish to receive guidance from someone who has already walked this path many times. * Adjunct Professor of Law, IU Robert H. McKinney School of Law; Law Office of Jeff Cardella. J.D. IU Robert H. McKinney School of Law. 2018] UPWARD! HIGHER 439 REACTION TO BAKER, EXTERNSHIPS: TEACHING, PRACTICE, AND THE BUILDABLE HOUR JAMES STRICKLAND* In the summer of 2016, I served as a judicial extern at the Indiana Supreme Court. Coming from a prior career as a high school humanities teacher, this externship helped ease me into the legal profession with the same hands-on practicality of the apprenticeships dating to colonial times. But unlike these early training programs, my externship was designed to incorporate the types of guidance discussed in Professor Baker’s Externships: Teaching, Learning, Practice and the Buildable Hour. When I was teaching, I required my students to set goals early and often. And my externship required me to set goals to achieve by the end of the summer. Following my externship, I revisited these goals, assessing my success. These prospective and retrospective reflections gave me the chance to focus my work throughout the summer and consider how to better realize future goals. I gave a lot of feedback as a teacher. But because I have focused on taking the bar-tested courses frequently assessed through blindly-graded final exams, I have not always received consistent and constructive criticism in law school. Not so in my externship: much proverbial red ink was spilled upon my work product. And I became a better writer for it. After years of giving gentle, unilateral feedback, my externship helped prepare me to receive the candid, recursive feedback common in the legal field. The first day of a new class, my students and I would establish behavioral boundaries for all of us to follow. Similarly, my externship class required analysis of observed good—and bad—lawyering attributes. And not only that, this assignment asked me to think about how these characteristics applied to me: Did I exhibit any of the bad qualities? Could I improve in the good? The traits I identified, both exemplary and cautionary, will forever shape my conduct as a lawyer. And starting this fall, I will serve as a judicial law clerk for Justice Mark S. Massa—in the very chambers where I served as an extern. As I begin to prepare for that role, I put great thought into how I too can mentor externs to set goals, receive constructive feedback, and self-reflect. The recursive process starts anew. * J.D. Candidate, 2018, Indiana University Robert H. McKinney School of Law, Indianapolis, Indiana; B.A., 2011, University of Missouri, Columbia, Missouri. Special thanks to those who mentored me during my judicial externship: Chief Justice Loretta H. Rush; Justice Mark S. Massa; Professor Joel M. Schumm; judicial law clerks Jeffrey Dunn, Dylan Pittman, Alex Preller, Leah Seigel, and Amanda Terrell; and judicial assistants Amber Holland and Jessica Strange. 440 INDIANA LAW REVIEW [Vol. 51:413 GIVING STUDENTS A SEAT AT THE TABLE: USING TEAM-BASED LEARNING IN CRIMINAL LAW SHAWN MARIE BOYNE* I. INTRODUCTION: WHY USE TEAM-BASED LEARNING? Perhaps in no other time since the release of the MacCrate1 and Carnegie Foundation2 Reports has the need for innovation in legal education been so critical. Since 2010, law schools have faced challenges meeting their enrollment targets as the total number of students entering law school has dropped.3 With a drop in applications, many law schools have seen the traditional metrics of LSAT scores and median undergraduate G.P.A.s drop as schools have struggled to fill classroom seats.4 Nationally, the percentage of first-year law students with scores of 149 or lower on the LSAT rose from 14.2 percent in 2010 to 22.5 percent in * Professor of Law, Indiana University Robert H. McKinney School of Law. I would like to thank Doug Jerolimov of the IUPUI Center for Teaching and Learning for conducting focus groups of my students as well as Tracey Birdwell who leads Indiana University’s Mosaic Active Learning Initiative. Hats off to my colleague Max Huffman for organizing McKinney’s Annual Upward! Teaching Symposium (“the Burning Man of law teaching”). 1. See generally ROBERT MACCRATE ET AL., LEGAL EDUCATION AND PROFESSIONAL DEVELOPMENT--AN EDUCATIONAL CONTINUUM (Am. Bar Ass’n Sect. of Legal Educ. & Admissions to the Bar 1992), available at https://www.americanbar.org/content/dam/aba/ publications/misc/legal_education/2013_legal_education_and_professional_development_macc rate_report).authcheckdam.pdf [https://perma.cc/X9KH-MMRK]. 2. See generally WILLIAM M. SULLIVAN ET AL., EDUCATING LAWYERS: PREPARATION FOR THE PROFESSION OF LAW (Carnegie Found. for the Advancement of Teaching 2007), available at http://archive.carnegiefoundation.org/pdfs/elibrary/elibrary_pdf_632.pdf [https://perma.cc/Q44W- QETB] (summary). 3. The number of individuals taking the LSAT exam peaked in 2009-10 at 171,500. At the low point since then, testing year 2016-17, 109,400 individuals took the LSAT exam. See LSAC End-of-Year Summary: LSATs Administered & Credential Assembly Service Registrations, LSAC, https://www.lsac.org/lsacresources/data/lsac-eoy [https://perma.cc/Z7T4-6H45] (last visited May 22, 2018). Reason exists to believe the trend is reversing, with increasing numbers of high scoring LSATs and increasing numbers of law school applications. See Karen Sloan, Law School Applications Are Up, Especially Among High LSAT Scorers, LAW.COM (Apr. 6, 2018), https://www.law.com/2018/04/06/law-school-applications-are-up-especially-among-high-lsat- scorers/?slreturn=20180527170710 [https://perma.cc/J3WU-AF9S] (last visited July 30, 2018). 4. Cf. Ry Rivard, Lowering the Bar, INSIDE HIGHER ED. (Jan. 16, 2015), https://www.insidehighered.com/news/2015/01/16/law-schools-compete-students-many-may-not- have-admitted-past [https://perma.cc/WN4E-65GD ] (interpreting ambiguous data to mean “the vast majority of law schools are . . . admitting less-qualified students”). 2018] UPWARD! HIGHER 441 2013.5 Unsurprisingly, as it has become easier to be admitted to law school, bar passage rates have dropped.6 Faced with declining bar passage rates, many law schools have added academic success programs that offer students assistance in preparing to take the bar exam.7 Seeking a way to address these challenges in the 1L year, in 2016 I redesigned my 1L criminal law class using Team-Based Learning (TBL). First introduced at the undergraduate level more than twenty years ago by Professor Larry Michaelsen, TBL is now used in a wide variety of disciplines.8 The appeal of TBL is that it structures classroom learning to “go beyond simply ‘covering content’ and focus[es] on ensuring that students have the opportunity to practice using course concepts to solve problems.”9 Teaching a large 1L class with close to 100 students, TBL offered me a way to ensure that all my students were engaged with analyzing and applying the course material, rather than simply writing down notes. In a TBL classroom, lecture time is minimal, as students spend the bulk of the class time working in groups on problem-based exercises. The instructor’s main role is not to lecture, but rather to design a constructive learning environment. The primacy of group learning in TBL classrooms leads to higher levels of student engagement,10 increased excitement in the classroom,11 5. Jerry Organ, The Composition of Graduating Classes of Law Students—2013-2016 —Part One, LEGAL WHITEBOARD (Dec. 29, 2014), http://lawprofessors.typepad.com/legalwhiteboard/ 2014/12/the-composition-of-graduating-classes-of-law-students-2013-2016-part-one-.html [https://perma.cc/8HXW-J6QL] (reporting the percentage of law school matriculants with LSAT scores in the below-150 category from 2010 to 2013); see also Jonathan Berr, Are law school admission standards slipping?, CBS NEWS (Jan. 21, 2015, 5:45 AM), https://www.cbsnews.com/ n e w s / la w -s c h o o ls -a r e -a d m it t in g -s t u d e n t s - t h e y - w o u l d - h a v e - r e j e c t e d -ye a r s -a g o / [https://perma.cc/7FH2-HX8G]. 6. See Mark Hansen, What do falling bar-passage rates mean for legal education—and the future of the profession?, ABA J. (Sept. 2016), http://www.abajournal.com/magazine/article/legal_ education_bar_exam_passage [(reporting nationwide decline in Multistate Bar Exam scores and low overall bar passage rates at selectively identified schools). 7. See CATHERINE L. CARPENTER ET AL., A SURVEY OF LAW SCHOOL CURRICULA 2002- 2010, at 16 (Am. Bar Ass’n Sect. of Legal Educ. & Admissions to the Bar 2012), available at https://www.americanbar.org/content/dam/aba/publications/misc/legal_education/2012_survey_ o f _ l a w _ s c h o o l _ c u r r ic u la _ 2 0 0 2 _ 2 0 1 0 _ e x e c u t i v e _ s u m m a r y . a u t h c h e c k d a m . p d f [https://perma.cc/KG39-RKX8] (stating that, by 2010, nearly half of the schools surveyed had added a bar preparation course to their curriculum). 8. Larry K. Michaelsen & Michael Sweet, The Essential Elements of Team-Based Learning, in TEAM-BASED LEARNING: SMALL-GROUP LEARNING’S NEXT BIG STEP 7-27 (2008). 9. Id. at 7. 10. See Eun-Kyung Chung et al., The Effect of Team-Based Learning in Medical Ethics Education, 31 MED. TCHR. 1013, 1013-17 (2009) (reporting results of switching to TBL in a medical ethics curriculum). 11. See generally EA Andersen et al., Implementing Team Based Learning in Large Classes: Nurse Educators’ Experiences, 8 INT’L J. NURSING EDUC. SCHOLARSHIP 1-16 (2011). 442 INDIANA LAW REVIEW [Vol. 51:413 and higher level of performance on standardized exams.12 Although use of TBL in law school classrooms is not yet widespread, other faculty have pioneered its use.13 II. BASIC STRUCTURE OF TBL Because the basic structure and philosophy of TBL courses has been described in detail elsewhere, in this essay I will only briefly highlight those elements as I explain how I structured my course.14 There are four essential elements to TBL: 1. Teams must be properly formed and managed. 2. Students must be motivated to come to class prepared. 3. Students must learn to use course concepts to solve problems. 4. Students must be accountable. A. Forming Teams Teams do the heavy lifting in a TBL classroom. In order for the group output to outperform the sum of the individuals, the group members learn how to listen to each other as well as how to resolve disagreements. It is also important that teams be diverse. I broke my class into five- and six-person teams using our law school’s course management software. While many experts in TBL recommend taking the time to form heterogeneous groups,15 I found that with a large class size, randomized selection worked fine.16 It also was easy to explain the group composition to students. At the same time, I intentionally spent time on group formation exercises such as practicing active learning and developing facilitation skills.17 12. Cf. Nancy A. Letassy et al., Using Team-Based Learning in an Endocrine Module Taught Across Two Campuses, 72 AM. J. PHARM. EDUC. 103 (2008) (evaluating effects of TBL on unit examinations to find five percent improvement in overall scores). 13. See, e.g., Sophie M. Sparrow & Margaret Sova McCabe, Team-Based Learning in Law, 18 J. LEG. WRITING INST. 153 (2012) (providing guidance for implementing team-based learning in a doctrinal law course); see Janet Weinstein et al., Teaching Teamwork to Law Students, 63 J. LEGAL EDUC. 36 (2013) (discussing the value of team-based learning in teaching teamwork in law school); see Anne E. Mullins, Team-Based Learning: Innovative Pedagogy in Legal Writing, 49 U.S.F.L. REV. F. 53 (2015) (discussing team-based learning in a legal writing curriculum). 14. See, e.g., JIM SIBLEY & PETER OSTAFICHUK, GETTING STARTED WITH TEAM-BASED LEARNING (2014) (discussing the “essential elements of team-based learning”). 15. See id. at 66; see also Sparrow & McCabe, supra note 13, at 196-97. 16. See Mullins, supra note 13, at 55 (stating that collecting demographic and other information from students may not facilitate strategic choices in a 1L writing class). 17. I found it helpful to show two YouTube videos produced by UNH’s Franklin Pierce School of Law that simulate a dysfunctional student group and a functional group in action discussing a group quiz. See Piercelaw, TBL in Law School - Simulated Student Quiz Discussion, YouTube (May 4, 2010), https://www.youtube.com/watch?v=oEK1DKBr3vU (dysfunctional 2018] UPWARD! HIGHER 443 One mistake that I made was to rotate teams twice during the course. Not only did that entail a great deal of administrative work, it also did not give students enough time to “regroup.” It takes time for students to get to know each other as well as to learn how to actively listen to and learn from each other. By switching teams twice during the semester, students had to spend time learning how to work with a new group of people. Because course coverage is an issue in bar courses, devoting additional time to forming new teams during the semester leaves less time to review application exercises. B. Student Motivation & Preparation “The quizzes forced me to stay on top of the reading.”18 In TBL courses, the content of a course is broken down into modules. Each module begins with a sequence known as the Readiness Assurance Process (RAP), which is structured to ensure that students come to class with a basic foundational knowledge of the material. TBL motivates students to read and study the assigned reading material by starting each module with a multiple choice quiz called an Individual Readiness Assessment Test (iRAT). After each student completes the IRAT, each team then takes the same quiz as a team (the Team Readiness Assurance Test, or tRAT), recording their answers on a scratch- off card. If the group scratches off an incorrect answer as its initial choice, the team is able to select another answer until the team selects the correct answer. The team, however, will only receive reduced credit for an incorrect initial answer choice. In an ideal scenario, as the members of each group discuss their answers to the quiz questions, students will explain and debate potential answers “unleashing the power of social learning and immediate focused feedback.”19 After the groups have submitted their group quizzes, groups then simultaneously report each of their initial answers to the quiz. I accomplished this by giving each group large laminated signs (A, B, C, D, and E) that they would hold up as I reread each question out loud. This simultaneous reporting allows students to see how their group performed relative to other groups and, when multiple groups answered the same question incorrectly, it signals questions that the instructor must take time to review. The review can be a mini-lecture of the most difficult concepts. It is very important that the quizzes target only a basic understanding of the material. Ideally the quiz questions should be targeted at the level of the “table of contents” rather than at the “index level.” The multiple choice questions that I developed were modeled more at the level of baby bar exam questions, too complex for the exercise.20 As a result, there were frequent appeals, and I found group) and https://www.youtube.com/watch?v=b6gOIkyax-8 (functional group). 18. Anonymous student comments were taken from the notes of focus groups conducted by Douglas Jerolimov, Ph.D. 19. SIBLEY & OSTAFICHUK, supra note 14, at 11. 20. According to Sibley & Ostafichuk, this is a common mistake made by beginning TBL 444 INDIANA LAW REVIEW [Vol. 51:413 myself cutting off students who wanted to use class time to argue for points. The entire Readiness Assurance Process (RAP) from the iRAT through the mini- lecture should take approximately one to one and a half hours. C. Students Must Learn to Use Course Concepts to Solve Problems “The group work and the discussions have, by far, assisted my learning the very most. I learn a great deal by working through problems and listening to how my classmates think and go about solving legal issues.”21 Although the fact that students must invest time in preparing for the RAP is key to learning gains made in TBL classrooms, the greatest benefit that I observed came from the application exercises that follow the RAP process. Typically, if one class period is devoted to the RAP process, the next three to five classes are devoted to group application activities. The application exercises require the group to work on significant problems that engage the whole team. Ideally, the groups should work on a complex problem with incomplete information where the diverse perspectives of different group members will be an asset.22 In my criminal law class, the types of activities that worked out the best included: 1. Giving students a prosecutor’s case file of a case and asking the students to craft sentencing recommendations for a defendant using the different theories of punishment. 2. Creating a complex factual scenario involving a homicide. Require students to: determine what level of homicide should be charged, what facts the prosecutor should use at trial to show the elements of the crime, as well as what facts the defense should highlight to reduce the type of homicide. D. Students Must Be Accountable While the RAP process encourages students to study for the quizzes, by itself that process does not ensure that all students participate in the discussion of the application exercises. In a typical TBL course, students complete a peer review of their team members.23 Because I changed teams twice throughout the semester, each student was peer-reviewed three times during the course. The two main problems that I encountered with the peer review process were that 1) in the high performing teams, students tended to give each other the same numerical score and 2) when students wrote comments on the review forms, many of those teachers. See supra note 14, at 109-10. 21. See supra note 18. 22. See SIBLEY & OSTAFICHUK, supra note 14, at 119. 23. See id. at 148-49. 2018] UPWARD! HIGHER 445 comments were accusatory, rather than constructive.24 Because practicing lawyers must often learn to give feedback to their support staff and other litigation team members, I am not going to scrap the peer review process, but I am searching for ways to refine it so that the feedback is more helpful. CONCLUSION: MAJOR LESSONS LEARNED Switching from the Socratic Method to TBL in a 1L class is a huge cultural shift. Rather than playing the “sage on the stage” role, the professor’s primary role is as a facilitator. Some students will inevitably resist the change especially if this is the only TBL course in the 1L curriculum. Despite the tremendous effort it takes to create a TBL course, the effort is well worth it. There were a number of days when the application exercises clicked and I was buoyed by the energy and engagement of my students. As I prepare for Round 2, there are three main changes I will be making to my course. First, because most of our 1L instructors use PowerPoints in the classroom, switching to a no-lecture course can be daunting for the students who view lecture notes as a form of security that they are learning something. Next year, I will post short video lectures on each unit using PowerPoint slides. These eight to ten minute lectures will highlight the key concepts in each chapter of the text. Second, I am going to make the quizzes more straightforward so that less class time will be devoted to arguing alternative answers. If students wish to file an appeal, I will give them a form at the end of the class period. This will give me more time in class to review and discuss the key points from the application exercises. I may also make a short PowerPoint to review the application exercise answers. Finally, I intend to “sell” the benefits of TBL throughout the semester, rather than just in the first class of the course. There are numerous studies that show that TBL increases students’ performance on standardized tests. Although I referenced some of those studies in attempting to “sell” my students on TBL last year, the real time to “sell” TBL is when a student or a group makes a breakthrough in understanding the material during an application exercise. Law school professors have been criticized for giving scant constructive feedback during the semester. Because TBL’s application exercises give students an opportunity to apply the knowledge learned before the final exam, they offer a great opportunity to deliver formative feedback at the same time as underscoring TBL’s benefits. If you are searching for a way to improve student learning outcomes and add excitement to your law school classroom, I invite you to convert your class to the TBL model! 24. I identified high-performing teams by the scores on the tRAT. 446 INDIANA LAW REVIEW [Vol. 51:413 REACTION TO BOYNE, GIVING STUDENTS A SEAT AT THE TABLE YVONNE DUTTON* In her contribution, Giving Students a Seat at the Table: Using Team-Based Learning in Criminal Law, Professor Boyne persuasively demonstrates how law teachers might employ team-based learning (TBL) in their classrooms to enhance student learning. For many years, the mainstay of law school pedagogy has been a combination of lecture and the Socratic Method questioning of students. While those are sound teaching techniques, declining bar passage rates nationwide provide one good reason for law professors to explore additional teaching methods. TBL, as Professor Boyne explains, goes beyond covering course content and allows students the opportunity to actually practice applying course concepts to solve lawyerly problems. Of course, with Socratic method questioning, the student “on call” will have to apply course concepts in response to the professor’s questions. With TBL, however, all students are essentially on call all the time: none has the luxury of “relaxing” while the “on call” student responds. In a TBL course, all students complete a quiz prior to class that tests their understanding of the reading. The professor is then able to focus her lecture time on any identifiable areas of difficulty. The remaining class time is reserved for students to use the knowledge they have gained from the reading and the in-class lecture. Specifically, they work in groups to solve a problem-based exercise. For example, Professor Boyne created a complex factual scenario involving a killing and required students to play the role of prosecutor and (1) decide the level of homicide to charge; (2) identify the facts they would use at trial to demonstrate that the elements of that homicide crime had been satisfied; and (3) identify the facts the defense would highlight to argue the prosecution had failed to meet its burden of proof. The above, of course, is only a brief summary of how the TBL classroom works. Professor Boyne’s article is rich in detail to assist any innovative law professor willing to explore new teaching territory. Studies show that the group learning in TBL classrooms leads to greater student engagement and interest. For law schools concerned about their bar passage rates, another advantage of TBL should stand out: studies show that group learning in TBL classrooms leads to a higher level of performance on standardized exams. * Professor of Law, Dean’s Fellow, Grimes Fellow, IU Robert H. McKinney School of Law. 2018] UPWARD! HIGHER 447 ASSESSING LAW STUDENTS MARGARET RYZNAR* INTRODUCTION As assessments take a larger role in both live and online courses per ABA regulations and best practices, this contribution advances the understanding of law students’ perspectives regarding assessment, offering empirical data on their assessment preferences in an online doctrinal course. The data used for this contribution consists of the mid-semester anonymous course evaluations of approximately 280 total students. The results provide new insight into law students’ assessment preferences and perspectives, which can be used to build assessments maximizing their learning and engagement. This contribution, part of an IU McKinney School of Law symposium on teaching and learning, offers an empirical analysis of student preferences on assessment methods. The data comes from my Online Trusts & Estates course, which is my contribution to McKinney Law Online after teaching it in a live classroom for years.1 Trusts & Estates is an important course in the law school curriculum. Most state bar exams test the subject, it is a staple in solo practitioner work, and all law students should be literate in planning their own estate. Thus, it is important that students learn the subject well, particularly in the autonomous setting of an online course. The primary way to measure student performance in a course throughout the semester—as well as to provide feedback to students and engage them—is through regular assessment. Assessments in legal education have received attention on a much broader scale. For example, the ABA has increased its emphasis on assessment.2 Additionally, the Carnegie Report has noted the critiques of traditional law school * Professor, Indiana University McKinney School of Law. Thanks to Yvonne Dutton and Max Huffman for conversations on the topic, as well as Christina L. Fisher and David Paul for superb analysis of the data, and to Julia Sanders for moderating the panel presentation at Upward! (Higher). 1. IU McKinney Law Online, MCKINNEY SCH. OF LAW, https://mckinneylaw.iu.edu/ courses/online-program/ [https://perma.cc/7YDM-FNBC] (last visited May 22, 2018). 2. SECTION OF LEGAL EDUCATION AND ADM ISSIONS TO THE BAR, AM. BAR ASS’N STANDARDS AND RULES OF PROCEDURE FOR APPROVAL OF LAW SCHOOLS, 2016-17, Standard 314, at 23 (2016) [hereinafter 2016 STANDARDS AND RULES] (“ASSESSMENT OF STUDENT LEARNING. A law school shall utilize both formative and summative assessment methods in its curriculum to measure and improve student learning and provide meaningful feedback to students.”); see also Mary A. Crossley & Lu-in Wang, Learning by Doing: An Experience with Outcomes Assessment, 41 U. TOL. L. REV. 269, 271-73 (2010) (giving an overview of the American Bar Association’s consideration of standards regarding assessment). 448 INDIANA LAW REVIEW [Vol. 51:413 assessment.3 Law student assessment and online education are closely linked. For example, the online learning course platforms that host online courses contain many built- in assessment tools allowing professors to ensure that their students learn the material correctly throughout the semester. Online professors have a particular incentive to use this technology to monitor and interact with students outside the classroom. However, these online platforms are also available for professors teaching live courses, and are being increasingly used as supplements to live courses. As assessments take a larger role in both live and online courses, this contribution advances the understanding of law students’ perspectives regarding assessment,4 offering empirical data on their assessment preferences in an online doctrinal course such as Trusts & Estates.5 Specifically, approximately 280 student responses to the following question have been coded: “What is your most AND least favorite type of online activity—discussion boards, quizzes, polls, or sample essays & answers? Please explain.” The results provide new insight into law students’ assessment preferences and perspectives, which can be used to build assessments maximizing their learning and engagement.6 These empirical results are indicative of general student preferences. This is because Trusts & Estates is a typical law school course, although there are some characteristics that make it unique. For example, it is a code-driven subject (i.e., governed by the probate and trust codes). As a result, it is rule-based. These features prompt Trusts & Estates students to learn a significant number of rules and black letter law, for which assessments are useful in ensuring that students have comprehended the material. 3. See WILLIAM M. SULLIVAN ET AL., EDUCATING LAWYERS: PREPARATION FOR THE PROFESSION OF LAW 167 (Carnegie Found. for the Advancement of Teaching 2007), [hereinafter Carnegie Report]. 4. Emily Zimmerman, What Do Law Students Want?: The Missing Piece of the Assessment Puzzle, 42 RUTGERS L.J. 1, 4 & n.9 (2010) (“Assessment impacts a student’s experience both in and after law school. In law school, grading can impact a student’s self-esteem and morale, what courses a student chooses to take, whether a student is able to participate in particular activities and receives certain honors while in law school, and whether a student is even eligible to remain in law school.”). 5. Id. at 2 (“Despite the importance of assessment in legal education, there is remarkably little empirical research that focuses on the perspectives of the constituency that is most immediately affected by assessment in legal education: law students.”). 6. Id. at 21-22 (“To the extent that different types of students have different assessment preferences, learning about students’ assessment preferences may also teach educators more about students than only their assessment preferences; students’ assessment preferences may be indicators of students’ wider attitudes towards and approaches to learning. Learning about students' assessment preferences also recognizes students’ active, central role in the learning process. In addition, students’ perceptions of assessment may actually influence how students learn. Therefore, learning about students’ perceptions of assessment is an integral part of studying students’ learning.”). 2018] UPWARD! HIGHER 449 While student preferences should not be entirely determinative in course design, they can contribute to course design because students are responsible for their own learning to some extent, particularly in an autonomous setting such as an online classroom. Furthermore, the data provide helpful glimpses into student engagement and motivation. II. ASSESSING LAW STUDENTS There are two categories of assessments, summative and formative.7 They differ in their purposes, with the goal of summative assessment being to grade or rank students. On the contrary, formative assessments exist to provide feedback to students and faculty on course performance so that students can develop their knowledge or skills and so that teachers have information about student performance and the effectiveness of instructional techniques.8 A. Traditional Methods of Assessing Law Students in Live Classrooms Historically, law school courses have often offered a summative assessment in the form of an exam.9 The most common methods of formative assessment for decades have been the Socratic method of questioning and other course discussions.10 These assessment methods are organic and fluid, as well as simple for the professor to administer because no student work needs to be read or graded. If a student answers incorrectly, it signals a problem to the professor—either the student has underprepared for the class or does not understand the material. The professor is able to call on other students to determine if there is a broader misunderstanding of the material, remedying it instantly through additional teaching. In live courses, there have been advancements in assessments.11 For example, technology has facilitated the use of clickers to quiz and poll students in real time in a live classroom.12 Professors may also assign readily available online lessons, 7. See, e.g., ROY STUCKEY ET AL., BEST PRACTICES FOR LEGAL EDUCATION: A VISION AND A ROAD MAP (2007). 8. See also Carol Springer Sargent & Andrea A. Curcio, Empirical Evidence That Formative Assessments Improve Final Exams, 61 J. LEGAL EDUC. 379, 380 (offering “evidence that formative assessments help law student performance on a cumulative final exam”). 9. See Carnegie Report, supra note 3, at 164 (“The end-of-semester essay examination holds a privileged, virtually iconic place in legal education.”). 10. See, e.g., Orin S. Kerr, The Decline of the Socratic Method at Harvard, 78 NEB. L. REV. 113 (1999). 11. Andrea A. Curcio, Moving in the Direction of Best Practices and the Carnegie Report: Reflections on Using Multiple Assessments in a Large-Section Doctrinal Course, 19 WIDENER L.J. 159 (2009). 12. See, e.g., Samantha A. Moppett, Control-Alt-Incomplete? Using Technology to Assess “Digital Natives”, 12 CHI.-KENT J. INTELL. PROP. 77, 104 (2013) (describing the use of clickers and other technology for assessing law students in a live classroom). 450 INDIANA LAW REVIEW [Vol. 51:413 which frequently have quizzes built into them.13 Finally, they may include a mid- term exam or leaving a sample final exam with an answer key in the library reserve. B. Innovations in Assessing Law Students Online The introduction of online learning platforms for courses, such as the Canvas course management system used at Indiana University, allows the introduction of many and various assessments for both online and live courses. Online teaching in particular requires more structured assessments as a substitute for in- class interaction. Professors may use these various assessments as summative assessments and grade them, or leave them ungraded as formative assessments. Professors may also comment to individual students based on their performance on the assessments, or comment to the entire class about trends in their performances. There are several ways to assess students on online platforms, the first of which is quizzes. Professors can create quizzes in various formats, such as multiple choice, fill-in-the blank, and even short essay. The online learning platform is often able to grade objective quizzes such as multiple choice, and can even offer students immediate feedback by revealing the correct answers and explanations upon a student’s submission of the quiz. The professor, meanwhile, receives a computer-generated report on student performance on that quiz once it closes. There are also more subjective assessment methods, such as discussion boards. In these, students post essay responses to a topic prompt, and interact with each other and the professor through the posts. Professors can make this assessment method more objective by creating a rubric for an appropriate response. There are also methods to poll students. In polls, students take a side in a debate and explain their reasoning in a written format. Finally, professors may provide self-assessment in the form of essay questions and answers. In this form of assessment, the students answer an essay question and compare their answers to either a rubric or a sample answer. In smaller courses, professors may more readily offer individual comments on such student work. III. EMPIRICAL DATA ON LAW STUDENT ASSESSMENT The online Trusts & Estates course that I have taught at McKinney has included ungraded formative assessment methods consisting of discussion boards, quizzes (mostly multiple choice), polls, and sample essays & answers. Students developed various opinions of these assessments, which were collected through an anonymous mid-semester evaluation with an open-ended question regarding the students’ most and least favorite assessment method, allowing students to 13. E.g., COMPUT ER AIDED LEGAL INSTRUCTION (CALI), https://www.cali.org/ [https://perma.cc/4TUP-KDJ7] (last visited May 22, 2018). 2018] UPWARD! HIGHER 451 elaborate on their opinions of the course. I have administered this mid-semester survey in each online Trusts & Estates course, which totaled three courses in three semesters with approximately 280 2L and 3L students in total. All of these mid-semester surveys have been coded and analyzed.14 The results are consistent. Each semester, the students reported that their favorite assessment was quizzes, and their least favorite was discussion boards: • Favorite Assessment— Quizzes • Least Favorite Assessment— Discussion Board • Fall 2016 • 61% • 58% • Spring 2016 • 69% • 50% • Spring 2015 • 52% • 32% The student comments further explained the students’ reasoning, revealing common themes: • My favorite online activity is the quizzes because I like checking my knowledge of the information with immediate feedback. • My favorite activities are the quizzes. It really helps me figure out whether or not I understand the material. • Most - the quizzes. I like the instant feedback and explanation provided for wrong answers. • I like quizzes best because they very quickly let me know if I am understanding the material or not. • My most favorite activity is doing the quizzes because it allows me to see what I know and don’t know about the material. • Most favorite - quizzes. I feel this is the best way to check my knowledge and make sure I'm actually picking up what I need to from the lecture.15 *** 14. Approval was received from the Institutional Review Board to use the student mid- semester evaluations in publications on online teaching. As part of the approval, it was agreed to report them anonymously and to remove any identifying information. 15. Other studies support the result I reach here, both in terms of student preference and in terms of measurable learning outcomes. See, e.g., Lorenzo Salas-Morera et al., Analysis of Online Quizzes as a Teaching and Assessment Tool, 2:1 J. TECH. & SCI. EDUC. (2012) (“The results obtained during the five years of experimentation using online quizzes shows that such quizzes have a proven positive influence on students' academic performance. Furthermore, surveys conducted at the end of each course revealed the high value students accord to use of online quizzes in course instruction.”) 452 INDIANA LAW REVIEW [Vol. 51:413 • Least Favorite: Discussion Board- I do not feel that this is very helpful overall. I am already thinking about the material at the time and everyone’s answers are pretty much the same. • My least favorite type of online activity is the discussion board because I feel like a lot of people say what has already been said. • I find the discussion boards very difficult. I often enter the conversation later in the week and struggle to find new thoughts or ideas to contribute. • [D]iscussion boards are my least favorite because it is difficult to come up with something original after the best answer has been repeated a few times. However, that’s not a bad in thing in that it lets me know I am on the right track. • Discussion board are my least favorite. I don’t feel like discussion boards help me see how much I understand the material. • Discussion boards are my least favorite type of online activity. I don’t feel that they help that much. • My least favorite activities are the polls and discussion boards. I feel like the answers get repetitive and it doesn’t help me comprehend anything better.16 CONCLUSION In sum, assessments have been increasingly taking center stage in the law school classroom, both online and live. Professors designing and teaching online can more easily integrate assessments in their courses due to technology, and they also have more incentive to use regular assessments to ensure that their students are keeping up with the material. There are various types of assessments. In law courses, there are some grey areas that come down to public policy and interpretation, and those questions are dealt with discussion boards, but it is important to ensure—especially in an online setting—that the students grasp the black letter law underlying these, and quizzes are a great way to do so. This is especially true in courses that are code-driven and problem-driven. Empirical data from approximately 280 students in three semesters of online Trusts & Estates at McKinney suggest that students prefer quizzes due to their immediate feedback. Students least prefer discussion boards because of repetition in student responses and the lack of immediate feedback. To comply with best practices and ABA expectations, law professors must confront how to best assess their students. Professors may want to consider, as one factor in course design, the empirical results presented in this contribution as they create such assessments in both their online and live courses. 16. Cf. JOHN ORLANDO, HOW TO EFFECTIVELY ASSESS ONLINE LEARNING 32 (Magna Pub. White Paper 2011), available at https://www.stjohns.edu/sites/default/files/documents/ir/ f63bd49dcf56481e9dbd6975cce6c792.pdf [https://perma.cc/S2TV-UJL6] (describing particular challenges of discussion boards in online teaching and learning). 2018] UPWARD! HIGHER 453 REACTION TO RYZNAR, ASSESSING LAW STUDENTS SHAWN MARIE BOYNE In her contribution, Professor Ryznar evaluates students’ preferences among various types of formative assessment practices. In teaching a large law school class online, Professor Ryznar faces the challenge of engaging students while stimulating their ability to become self-regulated learners. By using a variety of formative assessments throughout the semester, Professor Ryznar enables her students to assess their existing knowledge and competence. Drawing on data collected from mid-semester anonymous course evaluations of approximately 280 total students across three semesters, Ryznar notes that the vast majority of her students prefer taking online quizzes as opposed to participating in online discussion boards. The general gist of the student comments is that students like the immediate feedback that they receive from quizzes, while they do not feel that participating in the discussion board advances their understanding of the material. These results comport with prior studies of millennial students showing that millennials prefer interactive learning opportunities that incorporate regular assessments with immediate feedback. Although the data presented by Ryznar is a helpful starting point for understanding student engagement, it is does not help us to determine whether the assessment methods used in the course effectively improve student learning. While the student comments give us some indication that the quizzes enabled students to identify concepts that they had or had not learned, to improve our assessment measures, ultimately we need to gather data that goes beyond simply understanding student preferences. To be sure, student perceptions of a particular assessment practice may affect how students learn. However, the purpose of assessment is ultimately to evaluate “students’ attainment of defined learning outcomes.”1 It is difficult to determine the efficacy of an individual assessment tool without determining whether the tool helped students to achieve the course learning outcomes. To this end, we need to go beyond relying solely on student feedback and devise ways to gather data from students’ performance on the final exam. In particular, we need to develop rubrics that will enable us to determine whether students achieved the course learning outcomes. With this information in hand, we can go back and look at whether individual formative assessment practices impacted student progress towards those outcomes. By launching this type of inquiry, we can begin to understand exactly what our formative assessments are accomplishing. As one example, it would be helpful to know whether the quizzes are simply helping students to develop critical thinking skills at the lower levels of the 1. Victoria L. VanZandt, Creating Assessment Plans for Introductory Legal Research and Writing Courses, 16 J. LEGAL WRITING INST. 313, 320 (2010). 454 INDIANA LAW REVIEW [Vol. 51:413 revised Bloom’s taxonomy (“remembering” and “understanding”) or whether they aim higher at the levels of “applying,” “analyzing,” “evaluating,” and “creating.”2 It may be that quizzes are an effective method to test a student’s base knowledge of key concepts in the course, but that other types of exercises are necessary to improve students’ ability to move to the next level of thinking.3 Although “bar” courses like Trusts & Estates must expose students to key doctrinal concepts, the ultimate test is whether the course meaningfully improved students’ analytical capabilities.4 This process will entail a great deal of time and effort. While I commend Professor Ryznar for using formative assessments in her teaching, all of us, myself included, need to begin to examine ways to ask the more difficult questions about our teaching practices. 2. BENJAMIN S. BLOOM, A TAXONOMY OF EDUCATIONAL OBJECTIVES (2d. ed. 1956) (grading learning objectives). 3. See, e.g., Diane May, Using Scaffolding to Improve Student Learning in Legal Environment Courses, 31 J. LEGAL STUD. EDUC. 233, 237-38 (Summer 2014) (stating that scaffolding exercises help learners to meet the upper ends of their zones of proximal development). 4. See Todd E. Pettys, The Analytic Classroom, 60 BUFF. L. REV. 1255, 1307 (2012) (arguing that professors should use online activities to introduce students to basic doctrinal information but use in-class time to strengthen students’ analytic capacities). 2018] UPWARD! HIGHER 455 REAL WORLD PROBLEM-SOLVING IN THE DIGITAL CLASSROOM MARK L. SHOPE* INTRODUCTION The digital classroom is the perfect environment to connect students to the real world practice of law. Online tools can give teachers what they need to be more creative with respect to building an immersive, active learning experience for students. With the right approach, the digital classroom is arguably a better environment for creating a real world problem-solving learning experience, which will prepare students for the practice of law. This contribution will (1) briefly discuss the traditional law school classroom approach, (2) discuss the approach I took to my International Business Transactions class taught entirely online at McKinney, and (3) provide some concluding observations. I. TRADITIONAL LAW SCHOOL CLASSROOM Traditional law school pedagogy, in general, includes students reading cases and attending in-person class where a professor either lectures or engages in the Socratic Method with students. Legal skills and values are treated as separate subjects addressed in separate courses. The assessment for a traditional course is an issue-spotting exam that students, in response to a prompt, use as an opportunity to show the professor everything they know on the subject. Students usually just recall or recognize information required to respond to the prompt. There is little in the form of synthesis or evaluation—in part because there is limited time between the presentation of the prompt and the student response. Some new knowledge is constructed that reveals at least some student thinking that can be evaluated. There is usually no feedback other than a letter grade at the end of the course. Little of this model mirrors what happens in the real world practice of law.1 * Adjunct Professor of Law, IU Robert H. McKinney School of Law; Associate, Faegre Baker Daniels LLP. Thanks to Julia Sanders, Principal Instructional Designer with eDS, for tremendous course design support. 1. See generally Max Huffman, Online Learning Grows Up—And Heads to Law School, 49 IND. L. REV. 57, 60-61 (2015) (discussing limits on practical skills learning in traditional law school pedagogy). 456 INDIANA LAW REVIEW [Vol. 51:413 II. THE ONLINE CLASSROOM A. A Hypothetical Real-World Practice Experience The aim in my online course is to connect students to the real-world practice of law. In my course, students adopt professional roles and work to address authentic, complex legal problems. The students in my class engage in multiple role playing exercises to apply course concepts and are placed at the center of a real-world law practice. The student is a hypothetical associate attorney in the international practice at a local law firm, who is supervised by a very busy partner—just as students can expect to be if they pursue this career path. Students interact with the supervising partner, who is interacting with the general counsel of a fictional client with very realistic legal problems—problems you would see in the private practice of law. There are seven “modules” in my online classroom that each address a particular legal aspect of the international business world. Assignments are paced throughout the course at the end of each course module. In response to a prompt, students create a tangible product (including a “performance” in the example of the assignment below) that reveals their understanding of course concepts and their ability to comprehend, analyze, synthesize, and evaluate those concepts. There is a final exam, but it is worth only a small percentage of the final grade. My goal in creating this course was to make the students’ experience as real as possible. I use a problem-based textbook for the readings my class.2 Students watch videos that are less “lectures” and more of what I call “debriefings.” The major issues are discussed in these debriefings, but much of the discovery is left to the student. What I try to avoid is a situation where students turn on a video, put it on 2x speed so it goes by faster, and think that at the end of the video they know what is necessary to be successful in the course on that particular lecture topic. Students, I suspect, are left with more questions than answers after watching the debriefings, so they need to do further digging themselves. There are also PowerPoint slides that guide students through discrete issues with the readings. The students engage in an online discussion based on a prompt that relates in some way to the topics for each module. I make myself available during “virtual” office hours every week if students have questions. B. The Assignments The assignment for each module is worth around ten percent of the total grade in the class. These assignments are at the heart of the real-world approach in the class. Here is one example: 2. Cf. Myron Moskovitz, Beyond the Case Method: It’s Time to Teach with Problems, 42 J. LEGAL EDUC. 241 (1992) (outlining a problem-based approach to teaching traditional law school classes and contrasting it with the reigning “King Case Method”). 2018] UPWARD! HIGHER 457 Task You are still an associate at the same local law firm. General Counsel at Circuit Drone Concepts, LLC, realizing the dangers to their current payment terms with international customers, has decided to explore alternate forms of financing when international buyers are concerned. General Counsel and upper management at Circuit Drone Concepts, LLC know nothing about alternative financing options. They would like your law firm to give them a short presentation discussing their alternatives. Your supervising attorney is leaving for Dubai to visit an unrelated client and needs your help doing this. Read the email from Gerald and create PowerPoint slides and upload a video of your presentation. • The product should be a PowerPoint presentation and your video or audio narration/explanation. • You only need to discuss issues to the extent they are presented in the reading for this module. • The presentation should be no longer than 10 minutes. You must use VoiceThread[3] for this assignment. Due Date: February 12, 2017, 11:59 pm | Value: 10% From: Partner, Polly Date: Mon, January 30, 2017 at 4:24 AM Subject: FW: Help with Financing Presentation To: Associate, Alice Alice, OMG, im about to take off Dubai, no time. Plz take a look at Geradlas’ email below. I need ur help! Plz give them presentation in two weeks. PPT and vid. Thx. Polly Sent from my iPhone 3. An oral interactivity tool in use at Indiana University. 458 INDIANA LAW REVIEW [Vol. 51:413 ---------- Forwarded message ---------- From: Counsel, Gerald Date: Fri, January 27, 2017 at 1:24 PM Subject: Help with Financing Presentation To: Partner, Polly Polly, I’ve been discussing with senior management here at Circuit Drone Concepts, LLC about our financing options with regard to international sales. We don’t want to get stuck like last time with not getting paid and having to deal with our goods being stuck abroad. If you wouldn’t mind, we need you to give a short presentation regarding our options with regard to financing the international sale of goods (letter of credit, standby letter of credit, etc.). I know nothing about this. This presentation should include the different types of letters of credit, rules involved, the mechanics for each, and the issues/risks with each. Many thanks, Gerald Counsel Circuit Drone Concepts, LLC There are some examples of the real-world practice environment in this assignment, including: (1) the use of email as the mode of communication (including surprisingly casual communications); (2) associates doing the legwork; (3) the work product is a presentation to the client, an example of an authentic work product that a typical client may ask for; (4) students will have practice giving a presentation using VoiceThread (an online tool used to share audio/video presentations). Note that at the beginning of the class I describe the client and its activities. Some modules create new drama with respect to the client’s international activities, so students must take the particular client and its ongoing activities into consideration when completing each new assignment. Each assignment in my class asks for a different work product that can be applied in the real world and creates an engaging, active learning environment within the course. C. Evaluations I use a unique scoring rubric for each assignment.4 Feedback can be individualized, if requested, but I have found it more effective (and efficient) to 4. See generally Sophie M. Sparrow, Describing the Ball: Improve Teaching by Using Rubrics—Explicit Grading Criteria, 2004 MICH. ST. L. REV. 1 (explaining the value of scoring rubrics in communicating expectations, communicating feedback, and reducing the burden on faculty). 2018] UPWARD! HIGHER 459 provide global feedback to the class. This global feedback includes the major themes of what students did right and what needed improvement. Based on the global feedback, students can then reflect on their own assignment to better understand what they did well and what they did not do so well. The feedback can be used to make ongoing improvements as the course progresses.5 My rubric asks for real world skills to be demonstrated. The rubric for the particular assignment above asks for: Substantive Use of Accurate Information - What I look for here is an array of information that demonstrates the student’s ability to discern and work with the different financing options. This includes explanations of the different financing options that are complete, strategic, and indicate the pros/cons for each option, taking into consideration the client’s activities as a whole. Effective Slides - For this criterion, I am not looking for slides with visual properties exhibiting a high aesthetic quality. I am looking for slides that are coherent, have logical progression, are well-organized, and include main points. Clarity - Here I am looking for clear explanations of ideas that (1) clearly integrate the spoken presentation with the slides, and (2) demonstrate knowledge of key points. Style - Here I am looking for a fluent delivery that was well-paced, was clearly practiced, and fit the time requirement. These assignments are exactly the kind of products and content that attorneys in this field will need to be able to deliver. CONCLUDING OBSERVATIONS The tasks for this course were designed to assess student ability to apply knowledge and skills to real-world challenges. Except for some multiple choice questions on the final exam, all the tasks for this course required students to create a tangible product that reveals the depth of student knowledge on the subject. Student outcomes in the class were, in my opinion, impressive. Students have a solid grasp of what is going to be required of them upon graduation and entering the legal field. Students have not only acquired much substantive knowledge related to international business transactions, but they have learned what it takes to deliver quality work to clients in the private practice of law. The inclusion of online offerings in the law school curriculum can diversify 5. Cf. Terri LeClercq, Principle 4: Good Practice Gives Prompt Feedback, 3 J. LEGAL EDUC. 418, 421-24 (1999) (identifying features of feedback including prompt or delayed, formative or summative, general or specific). 460 INDIANA LAW REVIEW [Vol. 51:413 methods for delivering instruction in every classroom setting (both traditional and purely online). My class was purely online and asynchronous (except for my virtual office hours), but I also see value in utilizing the online platform in traditional and hybrid classes as well. The point here is being able to utilize the various online tools to diversify the delivery of instruction. A progressive law school education should include varying modes of instruction to meet students’ varying learning preferences and interests. These varying modes of instruction provide opportunities to apply learning beyond the classroom to solve real world problems. 2018] UPWARD! HIGHER 461 REACTION TO SHOPE, REAL WORLD PROBLEM-SOLVING IN THE DIGITAL CLASSROOM MARGARET RYZNAR Mark Shope’s submission Reflections on Real World Problem-Solving in the Digital Classroom is insightful, with the premise that online learning presents a unique opportunity to connect students to law practice. Indeed, some of the earliest online learning technology enabled a “flipped classroom,” where students were able to do activities in the live classroom after reviewing instructional materials online. Technology and course design has sufficiently advanced that practice activities can also occur online now, in addition to lecture videos and other instructional materials. In fact, technology has progressed so much in legal practice that many tasks performed by young lawyers are now best simulated in an online classroom. For example, young lawyers must write email memos and make video presentations, both of which Shope incorporates into his online course. They must work with electronic documents and correspondence. In sum, they must transition traditional lawyer tasks online to meet the demands of modern practice. Connecting students to law practice through an online course is not only Mark Shope’s stated goal, but also a guaranteed course outcome. This is facilitated by another feature of online education—the ability for the professor to assess students quickly and periodically. As the ABA increases its expectations of assessments in the live classroom, such a focus is already characteristic of online teaching. Indeed, without a live classroom in which to observe students, a professor teaching online must constantly review student work to ensure progress in the course. To do so, Mark Shope follows a rubric that conveys his standards for the various assignments at the end of each course module. This frequent expectation of work product not only mimics law practice, but also allows the professor to ensure that students find—and keep—themselves on the path to successful law practice. 462 INDIANA LAW REVIEW [Vol. 51:413 TEACHING INTERNATIONAL STUDENTS FRANK SULLIVAN, JR.* Five years ago, just before I began my teaching career at the IU McKinney School of Law, I made an argument for law schools and their faculties to recognize their LL.M. students “as far more than just revenue generators and [to] seek out opportunities for their international students to enrich the educational experiences of their J.D. students.”1 My article got a little attention2 but it was written by a then-outsider to the legal academy looking in. During ten semesters (plus five summers) on the inside, I have been forced to think not so much about using LL.M. students to benefit our J.D. students’ experience as about making sure that those LL.M. students themselves obtain the maximum benefit from their experiences in our classrooms. Here are some reflections on that subject, greatly informed by valuable observations from the dozen or so LL.M. students acknowledged in the footnote at the beginning of this essay. The essay contributes my views and experiences to a too-small literature on teaching international law students.3 * Professor of Practice, Indiana University Robert H. McKinney School of Law. Justice, Indiana Supreme Court (1993-2012). A.B., 1972, Dartmouth College; J.D., 1982, Indiana University Maurer School of Law; LL.M., 2001, University of Virginia School of Law. I am grateful to Linh Chu, Hanc Fei Han, Shang Jiang, Wei Li, Xing Liu, Tran Thi Bich Ngoc, Hao Nguyen, Omowole Omosehinmi, Suvd Tuul, Duong Quynh Vu, and Haixin Zhang whom I interviewed for this Article. For more information on the McKinney international programs and graduate programs, see Admission and Degrees: LL.M. Program, IND. UNIV. ROBERT H. MC KIN N E Y SC H . O F LAW, https://mckinneylaw.iu.edu/admissions/llm/index.html [https://perma.cc/HV56-79GC] (last visited May 22, 2018). 1. Frank Sullivan, Jr., International LL.M. Students: A Great Resource for U.S. Law Schools, 22 IND. INT’L & COMP. L. REV. 219, 235 (2012) (reviewing GEORGE E. EDWARDS, LL.M. ROADMAP: AN INTERNATIONAL STUDENT’S GUIDE TO U.S. LAW SCHOOL PROGRAMS (1st ed. 2011)). 2. See Deborah Burand, Crossing Borders to Create Value: Integrating International LL.M.’s into a Transactional Clinic, 19 LEWIS & CLARK L. REV. 441, 441, n.1 (2015) (quoting Sullivan, supra note 1). 3. See, e.g., Laura R. Dove & Natalie P. Bryant, Law in Translation: Challenges and Opportunities in Teaching International Students in Business Law and Legal Environment Courses, 33 J. LEGAL STUD. EDUC. 263, 263 (2016) (finding “one other article, written in 1991,” on the subject of teaching international students in a business law course). According to news reports, academic year 2016-2017 saw an all-time high number of international students in US colleges and universities, with Indiana in the top ten of states where these students undertake their studies. See, e.g., Rosanna Xia, Number of international students in U.S. colleges at an all-time high, and California is their top destination, L.A. NOW (Nov. 26, 2016, 4:00 AM), http://www.latimes.com/ local/lanow/la-me-study-abroad-students-20161124-story.html [https://perma.cc/7T5A-4WMM]. Not surprisingly, law faculty are very interested in “fostering inclusion of international students 2018] UPWARD! HIGHER 463 My own teaching portfolio consists of business and commercial law courses in which a handful of LL.M. students enroll each year (with the exception of Advanced Sales which, for the last two years, has consisted of approximately twenty-five percent international students). As such, my classrooms have a large to overwhelming majority of American J.D. students. I start each semester with a written welcome to the LL.M. students4 and the following oral admonition to the entire class as part of my first-day orientation: For me, one of the great things about this law school is a substantial number of students who come from other countries to study here each year. These are men and women of accomplishment, almost all of whom have completed law degrees from their home countries and have negotiated the language, financial, and other obstacles to travel to Indianapolis, Indiana, to study. We talk about the global legal marketplace and we have dozens of representatives of it under a roof. Make an extra effort to get to know some of these students; don’t wait for them to come to you; don’t wait for some special event to be organized. You may encounter a little language problem but it will be easily overcome. And think about how valuable it will be in the years ahead to know lawyers in Europe, Asia, Africa, and Latin America.5 Almost by definition, the international LL.M. student arrives in our classrooms with a degree in law from his or her home country—and often some experience in the practice of law.6 Nevertheless, language is the overarching challenge that each such student faces.7 As one of my students told me, “law is enrolled in post-JD programs as well as encouraging their interaction with JD students . . . .” Carole Silver, International students in US law schools: AALS discussions and the need for information, THE FACULTY LOUNGE (Jan. 10, 2017), http://www.thefacultylounge.org/2017/01/international- s t u d e n t s -in -u s -la w -s c h o o ls -a a ls -d is c u s s io n s -a n d -t h e -n e e d -f o r -in f o r m a t io n . h t m l [https://perma.cc/XZM3-KXTL]. 4. The first page of the syllabus in each of my courses contains the following language: “A special welcome to students from other countries: we are anxious for you to learn about this important dimension of American law and also to learn from you.” E.g., Frank Sullivan Jr., Advanced Sales—Syllabus (Spring 2018) (on file with author). 5. Frank Sullivan Jr., Lecture Notes (on file with author). In preparing this essay, one former LL.M. student wrote to tell me, “I still remember that you told our classmates to contact international students in our closely held business organization class. You encouraged American students to communicate initiatively with their international classmates, instead of waiting international students to go first. After that class, some local students came to say hey to me, and we became friends.” 6. See LL.M. Program: Eligibility Requirements for the LL.M. Program, IND. UNIV. ROBERT H. MCKINNEY SCH. OF LAW, https://mckinneylaw.iu.edu/admissions/llm/index.html [https://perma.cc/HV56-79GC] (last visited May 22, 2018) (applicants must have law degrees from the United States or another country or otherwise be licensed to practice law in a non-U.S. jurisdiction). 7. Cf. Dove & Bryant, supra note 3, at 268-69 (reporting on studies of English language 464 INDIANA LAW REVIEW [Vol. 51:413 a profession of language in a sense.” No matter how well a non-English speaking student is prepared to study in an American law school, the student says, he or she will immediately “confront the problem of legalese.” This student identified in particular what he called the “familiar word but different meaning” problem. A foreign student knows well the meaning of “fee” and “simple,” but no idea of the meaning of “fee simple”; the meaning of “false” and “imprisonment,” but not “false imprisonment.” The student said that the time required to consult the dictionary to make sense of these terms greatly reduced his reading speed. He suggested that a professor could be helpful in preparing a list of key words for the semester and provide it to the students, thereby saving students’ reading time. I have resolved to be deliberate and persistent in asking students to identify for me words and phrases that have caused them difficulty and to compile an online concordance with their meaning. This same student made another perceptive observation about an over-arching difficulty faced by international students in American law schools. He called this “the conflict of legal culture.” His point here is that in “the continental law system” of statutes in which many international students are trained, the “legal resource for a given case is almost certain.” This means that “foreign trained law students . . . want certain answers for a given legal phenomenon.” He perceptively contrasts this with the common law system, and especially the American common law system, where he observes “that a given case may be treated differently due to the application of different laws, common law and statutes, and within different jurisdiction. He recommended that professors warn students that they need to develop the habit of putting any “given case into different legal applications to understand the legal result.” Engaging international students in class is a matter of some sensitivity.8 I do it relentlessly.9 Several international students are enthusiastic about my approach. Says one, Let international students be a part of the class: as you may know, not many countries have the same educational culture like that of American, we are not taught to express our own opinion freely and even against our teachers, therefore, even though we saw American students confidently participate in the class, it is not easy for us to imitate them. Make us gain confidence gradually by letting us share our legal knowledge and encouraging answering some questions. Another echoes this point, saying that I should “[e]ncourage international students to interact with professors. Students from lecture-based education system may not know how to effectively participate into class discussion, especially problems international students encounter). 8. Id. at 270 (noting the differences in educational culture including classroom discussions in the United States contrasted with passive learning in some non-U.S. educational settings). 9. And, I have to say, often playfully: When I deploy an American colloquialism in class, e.g., “belt-and-suspenders,” I’ll call on an international student and demonstrate that American sayings that they are not familiar with are just as unfamiliar to their American counterparts! 2018] UPWARD! HIGHER 465 Chinese student, because we are not encouraged to do that in China.” A third student admonishes, “Give international students the chance to speak when they raise their hands in classes.” A fourth bluntly says, “Be tough with us.” But one student expresses a different attitude. This student says that he has two difficulties in dealing with the “cold call”: The first was about the reading. As an international student, sometimes I could not finish the reading even though I tried my best. Sometimes, I finished the reading the night before the class but forgot the details in the class the next day. I can memorize the content I read in one reading in my own language, but cannot memorize the content for several readings in English. The second was that I have a delayed response to English questions. Sometimes, I understood the question, and knew the answer to the question, but I could not respond as quickly as my American classmates. I felt panic when I was called. This further delayed my response. At that point in time, I really felt embarrassed. I believe that if professors could spare several minutes before the end of the class and tell the students what is the key issues of the next class and then ask the international students more about those issues, it would reduce the embarrassment and encourage class participation. I must also confess that several American students marked me down in their evaluations, believing that my calling on and interaction with international students made the international students uncomfortable and at least bordered on the insensitive. I certainly took these criticisms to heart and will try to be attuned to keeping my efforts to incorporate the international students into the course from crossing the line of propriety. My international students have giving me some suggestions for things I could do or do better: 1. Remember students’ names. “You do not need to pronounce the name correctly (we do not pronounce your name perfectly too) but it will make the students feel so much better,” says one.10 2. Talk with international students after classes. “For me, this is the best thing!,” a student told me. “I can ask professors any question I would like to do. The professors should also give students few comments about how they perform in the class.” Along these same lines, another says: International students have so many things that we do not know the answers during the class and do not dare to raise our hands. Therefore, 10. Adjunct Professors Joan Blackwell and Tom Blackwell got particular kudos from a student in this regard. “They tried to remember my name right from the first class,” the student said. 466 INDIANA LAW REVIEW [Vol. 51:413 we usually stay at the end of the class to ask about our concerns. Please be patient with us. We do know that professors also have to do research and other things, or have to go home after evening classes. But you are our only person that may help us out. We really appreciate all of your kind help. 3. Answer student email. 4. Speak slowly. “Sometimes,” one student told me, “it was hard for me to understand everything professors said during the class, and even when I understood them.” 5. A good outline from the professor can help international students to understand the big picture of the class. A student says, “I understand that law students should have prepared a good outline for every class by themselves. However, it is really difficult for international students to do so before they can fully understand what happened in the class, especially for LL.M. students and first year JD internationals.” My discussions with international students have driven home to me the need to help them adjust to a very different place, far from their homes. We can and should introduce them to our city and community and its living and studying environment. More broadly, we should acquaint them with state culture and history, maybe even organize opportunities to view a cultural or historic landmark. In addition, it is important for us to promote their awareness of the need for personal safety and self-protection. (Several years ago, one of our international students was killed in a car crash; it was just a terrible tragedy.) Lastly, the international students I interviewed for this Article were generous in their praise for many here at McKinney. Here are a few sample quotations: • For international students, Professor Nguyen is also a professor who helps us a lot. I would like to highlight her IP luncheon every Tuesday which is open for everyone and the topic covers many international subjects. Professor Nguyen also tried to host many other events for international student. • Professor Emmert always has a welcome party for international students. We appreciate that very much. • Professor Bravo’s class is also a very good one for international student. She truly helps us a lot, she always encourage us to get involved in more activities. • Joan Blackwell and Tom Blackwell always stay after class to answer literally any questions and talk to us. • Justice David is another professor who gives international students many chances to learn. The externship at the Indiana Supreme Court is a must-try experience. And Justice David is always very humorous which make us feel better every time we met him. 2018] UPWARD! HIGHER 467 • Finally, if you have a chance, please send my big thank-you to Miki [Pike Hamstra] and [Perfecto] Boyet [Caparas] for their support to us. • Former Associate Dean Claire Grove always hosted after-class events for LL.M. students to chat and would also invite JD students to join us. That gave us a lot of opportunities to chat and meet with fellow students. She helped me with a summer internship with Judge Baker at the Indiana Court of Appeals. That internship benefited me in many ways - legal writing, social networking with judges, clerks and other interns, and a great experience on my resume. • Professor Wilson was my faculty advisor and he went through the course list with me before school started. As an international student, we do not have many upperclassmen we could reach for before school starts. We all appreciate some “insider” information, and advice from faculty is tremendous. He showed me the souvenirs he got from China and that made me feel that someone knows my culture. That gave me great comfort. In a new environment, you attach to the people who understand and appreciate your culture and who have something in common with you. • Professor Edwards: The human rights program is such a great program. It gave me new perspectives in the human rights area. Professor Edwards helped place me with an externship at an NGO in DC. This program has brought U.S. students to different NGOs around the world. Making U.S. students international indirectly would make international students have more U.S. friends. A teacher’s interactions with students yield some incredibly gratifying experiences. One for me involved an international student, Shang Jiang, who, after completing an LL.M. at McKinney, went on to earn both a J.D. from the law school and an M.B.A. from the Kelley School of Business. While in Indianapolis, she was part of a group I organized to attend an Indianapolis Indians baseball game. After graduating, she moved to New York and sent me a picture of her and another recent graduate of our school attending a Yankees’ game. With it, she wrote: Never thought I would go to a Yankees' game that soon. I now understand most rules of baseball. Yeah! And I know there are differences between the National League and American League. Well, of course I responded, “Now that you know the difference between the American and National Leagues, what is your position on the designated hitter rule?” To which she responded: 468 INDIANA LAW REVIEW [Vol. 51:413 I take no position on the designated hitter rule. It is good to have division of labor, and it is not bad to see the pitcher bat. Which one is better, a specialized attorney or a general practice attorney, a specialty doctor or a family doctor? So hard to say. What is your position, Professor? My position is that we are lucky to have students from around the globe in our law schools and that we owe them our best. 2018] UPWARD! HIGHER 469 REFLECTION ON SULLIVAN, TEACHING INTERNATIONAL STUDENTS TIM WALTERS* In his article, Teaching International Students, Professor Sullivan describes an issue relevant for all members of the higher-education community: better facilitation of international-student learning in and out of the classroom. A classroom is not only affected by its size and the professor’s personality and approach to pedagogy, but also by students’ interactions. Professor Sullivan reminds us that our home culture shapes how we interact with others. One striking example was that international students are not taught to challenge or push back with what their professors teach. Personally, I have grown not only in my knowledge of the law but also my ability to reason through an argument by being forced to debate with the professor; in my mind, that process promotes poise and proficiency. U.S. students should be aware that international students are not taught to express their opinions, and should encourage their international peers to participate in class and answer questions when they can. Another reality Professor Sullivan addressed was the idea that “law is a profession of language.” Yet lawyers struggle with legalese that inundates situations with unnecessary complexity, even for those whose native tongue is English. If we are to make the learning environment conducive for international students, we need to work on communicating complex ideas in common words and not rely purely on our terms-of-art or legal jargon. We also should make the type of concordance Professor Sullivan suggests, and it would be helpful if it were accessible by U.S. students. If U.S. students were aware of particular areas of trouble, it could help facilitate constructive dialogue in the classroom, especially with team-based learning on the rise. Professor Sullivan suggests that the individual professors undertake the task of making such a concordance, but it should be a student-led committee. The committee could coordinate with the classes that have a high percentage of international students, conduct discussions with past students to discover the words and phrases that caused them the most trouble, compile the concordance, and then make it publicly available. Not only would this save a lot of administrative time and effort, but it could bring together the U.S. and international student communities in new ways. International students are not homogenous in their personalities, learning styles, and interests. But Professor Sullivan’s model strives to truly incorporate international students into the law-school community. The suggestions in his submission represent a step toward an inclusive community that values inherent contributions and not just financial ones. * J.D., 2018, Indiana University Robert H. McKinney School of Law; B.A., 2015, Wheaton College. 470 INDIANA LAW REVIEW [Vol. 51:413 SUMMARY AND CONCLUSION MAX HUFFMAN The contributors to this teaching and learning symposium sound a common theme in their otherwise highly diverse contributions. Each primary author (or pair of authors) offers an innovative approach to law teaching that combines (1) a need to produce a product that is relevant for a new and diverse student body with (2) a recognition of the trade-offs inherent in any productive enterprise. For example, Adams and I together reflect on the importance of lifelike experience in learning, and on making that experience available through modern and accessible pedagogical techniques. We draw the lesson that there can be too much of a good thing. Sullivan observes a primary outcome for teaching international students is preparing them to engage with their peers and their new learning—but balances that against concerns for embarrassment of, or unfairness toward, students working in a non-native language and a novel educational system. Similar trade-offs and solutions are found in each of the contributions. The Indiana Law Review editors’ thoughtfulness in bringing in reactions to the primary contributions, from full-time faculty colleagues, part-time faculty who are in law practice, and law students, gives a broad set of perspectives on how well the challenges were met in each case. Walters (a student contributor) echoes Sullivan’s observation that cultural norms influence classroom behavior. He continues with the suggestion that incorporating international students may call for change in the broader profession, “communicating complex ideas in common words and not rely purely on our terms-of-art or legal jargon.” Ryznar comments on Shope’s contribution, echoing his conclusions about the ability to produce practice-like experiences in the online classroom and even extending it. Connection to law practice, Ryznar concludes, is “a guaranteed course outcome” in Shope’s online course. Boyne, reacting to Ryznar’s contribution, challenges the law school and its faculty to “double down” on data-driven outcomes assessment, ensuring the effectiveness of our teaching and our students’ learning. Boyne’s comment on the need for data-driven outcomes assessment serves as an important asterisk on any report on the law school’s work. The larger lesson is in the importance of continually innovating, and evaluating those innovations, to address the dynamism in student demographics, the education market, and the professional environment. We learn these are not necessarily competing demands: online offerings meets student demand and, paraphrasing Professor Ryznar’s comment on Shope, prepares students for the tasks they will encounter on entering the profession. This compilation shows how an aggressively innovating faculty can keep itself ahead of the curve, always maintaining the educational mission at the front- and-center of its work.