Journal website: http://epaa.asu.edu/ojs/ Manuscript received: 9/20/2017 Facebook: /EPAAA Revisions received: 1/1/2018 Twitter: @epaa_aape Accepted: 1/12/2018 education policy analysis archives A peer-reviewed, independent, open access, multilingual journal Arizona State University Volume 26 Number 15 January 29, 2018 ISSN 1068-2341 Making Performance Assessments a Part of Accountability Billy Haun Virginia High School League (VHSL) United States Citation: Haun, B. (2018). Making performance assessments part of accountability. Education Policy Analysis Archives, 26(15). http://dx.doi.org/10.14507/epaa.26.3643 This article is part of the special issue, Redesigning Assessment and Accountability for Menaingful Student Learning, guest-edited by Soung Bae, Jon Snyder, and Elizabeth Leisy Stosich. Abstract: The purpose of this commentary is to describe recent efforts in Virginia to develop and use performance assessments, including the challenges that emerged during this process and key considerations for states that integrate performance assessment into their systems. Performance assessments can play an important role in preparing students for success and college and career, as described in Virginia’s Profile of a Virginia Graduate. Keywords: performance assessment; education policy; accountability, Virginia Creando evaluaciones de rendimiento que son parte de la responsabilidad Resumen: El propósito de este comentario es para describir los esfuerzos recientes en Virginia para desarrollar y utilizar evaluaciones de desempeño, incluyendo los desafíos que surgieron durante este proceso y consideraciones clave para los estados que integran la evaluación del desempeño en sus sistemas. Las evaluaciones de rendimiento pueden tener un papel importante en la preparación de los estudiantes para el éxito en la universidad y la carrera, como se describe en el Perfil de Virginia, una graduada de Virginia. Palabras clave: evaluación del desempeño; política educativa; responsabilidad; Virginia http://epaa.asu.edu/ojs/ http://dx.doi.org/10.14507/epaa.26.3643 Education Policy Analysis Archives Vol 26 No 15 SPECIAL ISSUE 2 Criando avaliações de desempenho que fazem parte da responsabilidade Resumo: O objetivo deste comentário é descrever os esforços recentes na Virgínia para desenvolver e usar avaliações de desempenho, incluindo os desafios que surgiram durante esse processo e considerações importantes para os estados que integram a avaliação de desempenho em seus sistemas. As avaliações de desempenho podem desempenhar um papel importante na preparação dos alunos para o sucesso na faculdade e carreira, conforme descrito no Virginia Profile, um graduado da Virgínia. Palavras-chave: avaliação de desempenho; política educacional; responsabilidade; Virgínia Making Performance Assessments a Part of Accountability Estimates project that by 2021 there will be over 500,000 new jobs in Virginia. Many of these jobs will be in the areas of science, technology, or the health care fields. Some of the jobs will need four year degrees, while many of these “middle level” jobs will require certification or only two year degrees. Virginia has made a commitment to make sure the students who graduate from our high schools have the skills and are prepared to compete for these jobs. Virginia’s Profile of a Virginia Graduate will describe the skills and dispositions students should attain during their K-12 school years. Action verbs are used to describe these skills and dispositions. The use of performance tasks is a method of assessment aligned with instruction that requires students to demonstrate, explore, attain, understand, align, and communicate. Students need to have opportunities to demonstrate what they know and use what they know to investigate and problem solve. Students cannot be provided these learning opportunities through multiple choice assessments alone. Many would categorize No Child Left Behind as a one-size-fits-all, watered-down, minimal standards approach that assessed with multiple choice assessments that allow for little or no creative teaching. ESEA has been in place and relatively unchanged for 18 years. ESSA has the potential to bring about changes and many of those changes can be accomplished with the use of performance tasks and assessments. Instruction can be individualized and scaffold to meet the learning styles and needs of students whether gifted and talented or students with learning disabilities or special needs. Performance tasks and assessments provide opportunities for collaborative and engaging learning, open-ended questions, and demonstrations of problem solving. Performance Assessment in Virginia While reading Stosich et al.’s (2018) How Do States Integrate Performance Assessments into Their Systems of Assessment? I reflected on the work Virginia did to begin the development of performance assessments. In 2014, the General Assembly passed legislation that replaced five state assessments with local assessments. The State Board of Education (BOE) created guidelines to support and guide this transition. While not mandating the use of performance assessments, the BOE encouraged their use. In the spring of 2014, Governor McAuliffe built on the work of the General Assembly by creating a Standards of Learning Innovation Committee. This committee was charged with making recommendations to the Board of Education and the General Assembly on ways to reform Standards of Learning, measures of student growth, and to innovate teaching. For the first two years, FY 2015 and FY 2016, the Virginia Department of Education was able to provide modest grants to each of Superintendent’s Regions. The purpose of these grants was twofold: 1) to provide fiscal resources to support professional development, and 2) to encourage collaboration and sharing among school divisions. Virginia’s 134 school divisions, which are similar Making performance assessments a part of accountability 3 to school districts in other states, were in very different places with performance assessment work and had varied resources. Challenges in Integrating Performance Assessment in Local Systems of Assessment A few divisions had started work on performance assessments and were actually implementing them as part of the curriculum, but most were just beginning. As we worked with the regions, several challenges divisions were facing came to light. One issue was the limited capacity of teacher training and knowledge. Many of Virginia’s teachers had little knowledge of performance assessments. A lot of teachers had attended public school in the era of No Child Left Behind and had actually experienced Standards of Learning assessments. They had been taught with multiple choice assessments being the model and had been trained to teach in the same way. These teachers were used to a multiple choice test-based accountability system and had never experienced performance assessments. Helping teachers learn how to use portfolios, projects or performance tasks was the first step for many divisions. A second challenge was helping teachers to “unpack” the Standards of Learning. To create quality performance assessments, teachers have to understand the standards. They were about to create assessments that were asking students to demonstrate not only what they know, but what they understand and can do. We were going to ask students to apply what they know. To do this, there has to be a deep understanding of the curriculum and standards. We were going to ask questions that had more than one correct answer. Once the quality performance assessments were created, the next challenge was scoring them. Many teachers had not created or used rubrics. Scoring student work with a rubric is much more challenging than grading a multiple choice assessment. With the implementation of performance tasks, the role of teachers changed. Most performance tasks require students to be more inquisitive, investigative, creative, and to work in teams. In classrooms where this type of learning is taking place, teachers become more of a moderator of learning rather than being the person imparting the knowledge. Resources, fiscal, infrastructure, and curriculum, are concerns in the development of performance tasks. While the work of developing and writing performance tasks does not require many fiscal resources, teacher time and professional development to build the capacity of teachers to complete that work can become costly. Staff infrastructure is a challenge for many divisions. Some divisions do not have administrative and instructional staff who can lead the work of developing performance tasks, so developing a knowledgeable training team can be difficult. The other staffing challenge for divisions is retaining staff, including superintendents and principals. Leadership changes often mean a change in priorities and focus for divisions and schools. There are great challenges facing states in the implementation of performance tasks as part of accountability. There will have to be some consistency in the performance task that gets implemented. Will states use common assessments and common use rubrics? How will the integrity of the assessment be protected? Who will score the performance assessments and how will the state guarantee consistency in scoring? All of this can happen. Advanced Placement consistently scores thousands of open-ended and essay assessments each year. It will be a matter of developing a similar type system at the state level. In 2016, the Virginia General Assembly adopted legislation that required the Board of Education to develop the Profile of a Virginia Graduate, the skills, knowledge, competencies and experiences students should attain in school that will make them life ready. When developing this Education Policy Analysis Archives Vol 26 No 15 SPECIAL ISSUE 4 profile, the 5 c’s of collaboration, communication, creativity, critical thinking, and citizenship should be incorporated. These skills will be implemented in revised graduation requirements and Virginia’s Standards of Accreditation. The need for the 5c’s and other dispositions, such as work place skills, application of knowledge, and an understanding of personal and civic responsibility, were identified as essential by stakeholders such colleges, universities, and employers. Performance assessments can play an important role in preparing students as defined by the Profile of a Virginia Graduate. Many of these skills and dispositions can and will be demonstrated by students in performance tasks. Performance assessments lend themselves to be more interdisciplinary, which creates opportunities for more authentic tasks. Performance tasks scored by rubrics also provide a better opportunity to award students credits based on the mastery of standards versus seat time. Key Considerations for the Use of Performance Assessment As states continue their work on integrating performance assessments, there are two considerations that people should keep in mind. The first is time. The process of educating and training teachers, creating high quality, rigorous performance assessments, and learning to score in a reliable and consistent manner will take time. States who are leading this work have taken between seven and 12 years to be in a place to have valid and reliable performance tasks that can be used as part of their accountability system. There are no short cuts in getting to this point. The second consideration is “balance.” In the late 1990s and the passage of No Child Left Behind, the country’s education system let the pendulum swing in one direction – multiple choice assessments as the only measure of accountability. It took 18 years to get reauthorization of ESSA, which gives states more control of their state accountability systems. While performance tasks are a tremendous method of assessing student achievement, every skill and standard cannot be measured by performance tasks. Proceed with caution and balance. About the Author Billy Haun Virginia High School League (VHSL) bhaun@vhsl.org Billy Haun is the Executive Director of the Virginia High School League, after years in education as a math teacher, varsity football coach, principal, Assistant Superintendent for Student Learning for Albemarle County Public Schools, and Chief Academic Officer for the state of Virginia. About the Guest Editors Elizabeth Leisy Stosich Fordham University estosich@fordham.edu Elizabeth Leisy Stosich is an Assistant Professor in Educational Leadership, Administration, and Policy at Fordham University. Previously, she was a Research and Policy Fellow at the Stanford Center for Opportunity Policy in Education. Her research interests include education policy, assessment and accountability, school and district leadership, school improvement, and teachers’ professional learning. mailto:bhaun@vhsl.org mailto:estosich@fordham.edu Making performance assessments a part of accountability 5 Soung Bae Stanford University soungb@stanford.edu Soung Bae is a Senior Learning Specialist and UDL Innovation Studio Manager at the Schwab Learning Center at Stanford University. Formerly, she was a Senior Research and Policy Analyst at the Stanford Center for Opportunity Policy in Education. Her research interests focus on school accountability, student engagement, and designing learning environments that appreciate and support learner variability. Jon Snyder Stanford University jdsnyder@stanford.edu Jon Snyder is the Executive Director of the Stanford Center for Opportunity Policy in Education (SCOPE). His research interests include teacher learning, conditions that support teacher learning, and the relationships between teacher and student learning. mailto:soungb@stanford.edu mailto:jdsnyder@stanford.edu Education Policy Analysis Archives Vol 26 No 15 SPECIAL ISSUE 6 SPECIAL ISSUE Redesigning Assessment and Accountability education policy analysis archives Volume 26 Number 15 January 29, 2018 ISSN 1068-2341 Readers are free to copy, display, and distribute this article, as long as the work is attributed to the author(s) and Education Policy Analysis Archives, it is distributed for non- commercial purposes only, and no alteration or transformation is made in th e work. More details of this Creative Commons license are available at http://creativecommons.org/licenses/by-nc-sa/3.0/. All other uses must be approved by the author(s) or EPAA. EPAA is published by the Mary Lou Fulton Institute and Graduate School of Education at Arizona State University Articles are indexed in CIRC (Clasificación Integrada de Revistas Científicas, Spain), DIALNET (Spain), Directory of Open Access Journals, EBSCO Education Research Complete, ERIC, Education Full Text (H.W. Wilson), QUALIS A1 (Brazil), SCImago Journal Rank; SCOPUS, SOCOLAR (China). Please send errata notes to Audrey Amrein-Beardsley at Audrey.beardsley@asu.edu Join EPAA’s Facebook community at https://www.facebook.com/EPAAAAPE and Twitter feed @epaa_aape. http://www.doaj.org/ mailto:Audrey.beardsley@asu.edu https://www.facebook.com/EPAAAAPE Making performance assessments a part of accountability 7 education policy analysis archives editorial board Lead Editor: Audrey Amrein-Beardsley (Arizona State University) Consulting Editor: Gustavo E. Fischman (Arizona State University) Associate Editors: David Carlson, Lauren Harris, Eugene Judson, Mirka Koro-Ljungberg, Scott Marley, Iveta Silova, Maria Teresa Tatto (Arizona State University) Cristina Alfaro San Diego State University Gene V Glass Arizona State University Susan L. Robertson Bristol University, UK Gary Anderson New York University Ronald Glass University of California, Santa Cruz Gloria M. Rodriguez University of California, Davis Michael W. Apple University of Wisconsin, Madison Jacob P. K. Gross University of Louisville R. Anthony Rolle University of Houston Jeff Bale OISE, University of Toronto, Canada Eric M. Haas WestEd A. G. Rud Washington State University Aaron Bevanot SUNY Albany Julian Vasquez Heilig California State University, Sacramento Patricia Sánchez University of University of Texas, San Antonio David C. Berliner Arizona State University Kimberly Kappler Hewitt University of North Carolina Greensboro Janelle Scott University of California, Berkeley Henry Braun Boston College Aimee Howley Ohio University Jack Schneider College of the Holy Cross Casey Cobb University of Connecticut Steve Klees University of Maryland Noah Sobe Loyola University Arnold Danzig San Jose State University Jaekyung Lee SUNY Buffalo Nelly P. Stromquist University of Maryland Linda Darling-Hammond Stanford University Jessica Nina Lester Indiana University Benjamin Superfine University of Illinois, Chicago Elizabeth H. DeBray University of Georgia Amanda E. Lewis University of Illinois, Chicago Adai Tefera Virginia Commonwealth University Chad d'Entremont Rennie Center for Education Research & Policy Chad R. Lochmiller Indiana University Tina Trujillo University of California, Berkeley John Diamond University of Wisconsin, Madison Christopher Lubienski Indiana University Federico R. Waitoller University of Illinois, Chicago Matthew Di Carlo Albert Shanker Institute Sarah Lubienski Indiana University Larisa Warhol University of Connecticut Sherman Dorn Arizona State University William J. Mathis University of Colorado, Boulder John Weathers University of Colorado, Colorado Springs Michael J. Dumas University of California, Berkeley Michele S. Moses University of Colorado, Boulder Kevin Welner University of Colorado, Boulder Kathy Escamilla University of Colorado, Boulder Julianne Moss Deakin University, Australia Terrence G. Wiley Center for Applied Linguistics Melissa Lynn Freeman Adams State College Sharon Nichols University of Texas, San Antonio John Willinsky Stanford University Rachael Gabriel University of Connecticut Eric Parsons University of Missouri-Columbia Jennifer R. Wolgemuth University of South Florida Amy Garrett Dikkers University of North Carolina, Wilmington Amanda U. Potterton University of Kentucky Kyo Yamashiro Claremont Graduate University Education Policy Analysis Archives Vol 26 No 15 SPECIAL ISSUE 8 archivos analíticos de políticas educativas consejo editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Editores Asociados: Armando Alcántara Santuario (Universidad Nacional Autónoma de México), Jason Beech (Universidad de San Andrés), Angelica Buendia (Metropolitan Autonomous University), Ezequiel Gomez Caride (Pontificia Universidad Católica Argentina), Antonio Luzon (Universidad de Granada), Angelica Buendia (Metropolitan Autonomous University), José Luis Ramírez (Universidad de Sonora) Claudio Almonacid Universidad Metropolitana de Ciencias de la Educación, Chile Juan Carlos González Faraco Universidad de Huelva, España Miriam Rodríguez Vargas Universidad Autónoma de Tamaulipas, México Miguel Ángel Arias Ortega Universidad Autónoma de la Ciudad de México María Clemente Linuesa Universidad de Salamanca, España José Gregorio Rodríguez Universidad Nacional de Colombia, Colombia Xavier Besalú Costa Universitat de Girona, España Jaume Martínez Bonafé Universitat de València, España Mario Rueda Beltrán Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México Xavier Bonal Sarro Universidad Autónoma de Barcelona, España Alejandro Márquez Jiménez Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México José Luis San Fabián Maroto Universidad de Oviedo, España Antonio Bolívar Boitia Universidad de Granada, España María Guadalupe Olivier Tellez, Universidad Pedagógica Nacional, México Jurjo Torres Santomé, Universidad de la Coruña, España José Joaquín Brunner Universidad Diego Portales, Chile Miguel Pereyra Universidad de Granada, España Yengny Marisol Silva Laya Universidad Iberoamericana, México Damián Canales Sánchez Instituto Nacional para la Evaluación de la Educación, México Mónica Pini Universidad Nacional de San Martín, Argentina Ernesto Treviño Ronzón Universidad Veracruzana, México Gabriela de la Cruz Flores Universidad Nacional Autónoma de México Omar Orlando Pulido Chaves Instituto para la Investigación Educativa y el Desarrollo Pedagógico (IDEP) Ernesto Treviño Villarreal Universidad Diego Portales Santiago, Chile Marco Antonio Delgado Fuentes Universidad Iberoamericana, México Paula Razquin Universidad de San Andrés, Argentina Antoni Verger Planells Universidad Autónoma de Barcelona, España Inés Dussel, DIE-CINVESTAV, México javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/819') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/820') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/4276') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/1609') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/825') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/797') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/555') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/823') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/798') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/814') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/2703') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/801') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/816') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/826') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/802') Making performance assessments a part of accountability 9 arquivos analíticos de políticas educativas conselho editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Editores Associados: Geovana Mendonça Lunardi Mendes (Universidade do Estado de Santa Catarina), Marcia Pletsch, Sandra Regina Sales (Universidade Federal Rural do Rio de Janeiro) Almerindo Afonso Universidade do Minho Portugal Alexandre Fernandez Vaz Universidade Federal de Santa Catarina, Brasil José Augusto Pacheco Universidade do Minho, Portugal Rosanna Maria Barros Sá Universidade do Algarve Portugal Regina Célia Linhares Hostins Universidade do Vale do Itajaí, Brasil Jane Paiva Universidade do Estado do Rio de Janeiro, Brasil Maria Helena Bonilla Universidade Federal da Bahia Brasil Alfredo Macedo Gomes Universidade Federal de Pernambuco Brasil Paulo Alberto Santos Vieira Universidade do Estado de Mato Grosso, Brasil Rosa Maria Bueno Fischer Universidade Federal do Rio Grande do Sul, Brasil Jefferson Mainardes Universidade Estadual de Ponta Grossa, Brasil Fabiany de Cássia Tavares Silva Universidade Federal do Mato Grosso do Sul, Brasil Alice Casimiro Lopes Universidade do Estado do Rio de Janeiro, Brasil Jader Janer Moreira Lopes Universidade Federal Fluminense e Universidade Federal de Juiz de Fora, Brasil António Teodoro Universidade Lusófona Portugal Suzana Feldens Schwertner Centro Universitário Univates Brasil Debora Nunes Universidade Federal do Rio Grande do Norte, Brasil Lílian do Valle Universidade do Estado do Rio de Janeiro, Brasil Flávia Miller Naethe Motta Universidade Federal Rural do Rio de Janeiro, Brasil Alda Junqueira Marin Pontifícia Universidade Católica de São Paulo, Brasil Alfredo Veiga-Neto Universidade Federal do Rio Grande do Sul, Brasil Dalila Andrade Oliveira Universidade Federal de Minas Gerais, Brasil