Discipline Policy Reform 1 _______________________________________________________________________________________________________________________ Global Education Review is a publication of The School of Education at Mercy College, New York. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License, permitting all non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. Citation: Robert, Catherine, & Okilwa, Nathern S. A. (2022). Discipline Policy Reform: A Review of Oakland USD following an OCR Investigation. Global Education Review, 9 (3), 1-18. Discipline Policy Reform: A Review of Oakland USD following an OCR Investigation Catherine Robert University of Texas at Arlington Nathern S. A. Okilwa University of Texas at San Antonio Abstract Research demonstrates harmful effects of disparate discipline practices on outcomes for students of color. The Office of Civil Rights (OCR), under the Obama administration, significantly increased investigations under the mandate of Title VI. In this study, we examine OCR’s investigation of Oakland Unified School District (OUSD) to establish the district’s response following findings of disproportionality for African American students. Current OUSD discipline data indicates persistent discipline disparities remain, yet OUSD has undertaken significant reform initiatives. These initiatives include establishing Restorative Justice programs, an African American Male Achievement Program (AAMA), and extensive internal review and revision of data collection practices for disciplinary incidents. Our findings indicate that challenges to this disciplinary reform effort include data quality, training, and the district’s continuing commitment to reform following the conclusion of OCR oversight. Descriptors OCR Review, OUSD, African American, discipline disparity, discipline policy The mission of the United States Department of Education, Office of Civil Rights (USDOE, OCR) is “to ensure equal access to education and to promote educational excellence throughout the nation through vigorous enforcement of civil rights” (USDOE, OCR, 2018). To live up to its mission, under the Obama administration, the Office of Civil Rights (OCR) increased efforts to enforce Title VI of the Civil Rights Act which prohibits discrimination against participation in any program or activity on the basis of race, color, or national origin (Lhamon, 2014; D’Orio, 2018). Using the Civil Rights Data Collection to identify areas of disparities between student groups, the OCR specifically considered school discipline rates by ethnicity as indicators of possible Title VI discrimination (Lhamon, 2014). The preponderance of evidence (e.g., Losen & Gillespie, 2012; Skiba et al., 2011; Smolkowski et al., 2016,) clearly demonstrates the harmful effects of disparate discipline practices on academic, social, and life outcomes of students, particularly minoritized students. Therefore, the OCR is a critical federal agency in monitoring educational inequalities such as discipline disparities. As Losen and Gillespie (2012) explain, students encountering persistent disciplinary consequences lose critical instructional time. Teachers may unintentionally use bias in their perceptions of appropriate classroom behaviors and unfairly penalize one group of students over another group for similar behaviors (Skiba et al., 2002; 2011). In using data to launch investigations, the OCR under the Obama administration reversed the previous administration’s practice in which investigations were initiated based solely on complaints. From 2009 through early 2012, the OCR launched 20 “proactive” investigations concerning discipline disparity (Office for Civil Rights, 2012). While civil rights 2 Global Education Review 9 (3) advocates hailed the work as overdue, school officials expressed concern over the broad reach of the probes and questioned the work as a potential “fishing expedition” (Maxwell, 2011, p. 2). In addition to increasing the number of investigations, the OCR also issued several guidance documents (such as Dear Colleague letters) and “technical-assistance activities” designed to clarify civil rights issues within school districts (Maxwell, 2011). The new guidelines created much confusion in districts as school officials struggled to both determine the legitimacy of the guidelines (as opposed to more stringent and clear regulations) and to alter district policies to match the changing OCR expectations (Pernell, 2015). As we await to see the Biden administration’s position on tackling disparities in educational outcomes, to include school discipline, we evaluate the impact of the significant efforts undertaken by the OCR during the Obama years to reduce school discipline disparity. The Authors (2018) previously evaluated the OCR’s first sweeping investigation of one large urban school district and found that the district complied with the activities described within the OCR’s compliance agreement, though many of the activities were still in the initial stages, six years following the signed agreement. For this study, the authors continue evaluation of the OCR’s compliance agreements specific to discipline disparity of African American students that were initiated and resolved during the Obama administration. Our goal is to examine the OCR investigation of Oakland Unified School District (OUSD) which was initiated in May 2012 to determine whether the OUSD “subjected African American students to discrimination on the basis of race by disciplining them more frequently and more harshly than similarly situated White students” (OCR, USDOE, 2012). Before the conclusion of the OCR’s investigation, OUSD elected to voluntarily enter into agreement with OCR thus committing to specific actions, such as to reform policies, practices, and activities to address the discipline disparity issue. While the full effect of changes to discipline policies will take several years to emerge within the data, this study first questions if OUSD fully engaged in reforms as agreed with the OCR. These initial efforts ideally begin the process of equalizing educational opportunities for all students. As much as the OCR’s investigations illuminated the disparate practices in OUSD, the responsibility for reducing inequalities ultimately requires commitment and response at the district level. Thus, to achieve the goal of this study, we raise three main questions: (1) In the OCR compliance review of Oakland USD concerning discipline disparities between African American and White students, what key elements of policy and practice did the OCR identify that needed reform? (2) What are the differences in discipline outcomes for African American students prior to and post OCR investigations of OUSD? and (3) For each key element identified by the OCR, what stage of reform has OUSD achieved? The rest of this paper is organized, first, by presenting a general overview of scholarly literature about school discipline disparity, followed by the policy change framework, methods section, findings, discussion, and lastly the conclusions. A Brief Overview of School Discipline Disparity Literature Evidence suggests that African American students are suspended significantly more than other students (Losen & Gillespie, 2012). For example, in their statewide study that examined suspension and expulsion of nearly one million public secondary school students (7- 12 grades) in Texas, Fabelo and colleagues (2011) found that African Americans and Discipline Policy Reform 3 3 students with disabilities were disproportionately suspended or expelled compared to their peers. Furthermore, the disparity is prevalent across the different grade configurations. For example, Skiba and colleagues (2011) found that African Americans are two times (elementary) and four times (middle school) more likely to be referred to the office for problem behaviors than their White peers following their study of office referrals of 364 elementary and middle schools. Similarly, in their high school study, Gregory and Weinstein (2008) established that African American students were overrepresented in the high school discipline referrals. Even in the case of well-documented, comprehensive disciplinary practice reforms in Syracuse, Denver, and Cleveland, substantial racial disparities continue to exist in these districts (Gregory & Fergus, 2017). Gregory and Fergus (2017) signal some optimism that while African American students “continue to receive harsher sanctions for similar misconduct” (p. 125), “substantially fewer students were excluded from instruction for discipline infractions” (p. 126). The persistent pattern of harsher sanctions for African American students is further illustrated by Okonofua and Eberhardt (2015) in their study that demonstrates how racial stereotypes contribute to teachers’ increased negative responses. Given clear patterns of discipline disparity, existing research demonstrating decreased student achievement among suspended students (Arcia, 2006) contributes to our understanding of lower academic outcomes of African American students. While the pattern of the “school-to-prison pipeline” (Okilwa et al., 2017; Skiba et al., 2011) documents the deleterious effects inherent within discipline disparity, there is less longitudinal research concerning discipline policy reform efforts. However, there is growing evidence schools are implementing programmatic approaches in an effort to address discipline disparity. In the following section we highlight two interventions, School-Wide Positive Behavior Supports (SWPBS) and Restorative Justice, which are commonly implemented with the goal of keeping students in school rather than suspension. Techniques Used in Addressing Discipline Disparity School-Wide Positive Behavior Supports. Several districts implementing School-Wide Positive Behavior Supports (also referred to interchangeably in the literature as Positive Behavior Interventions and Supports, or PBIS) (Sugai & Horner, 2009) over a period of many years have added promising practices and interventions demonstrating lowered rates of referrals for African American students. Particularly, schools in states such as Maryland, Illinois, North Carolina, and Oregon are worth close examination (Algozzine et al., 2010; Bradshaw, Koth et al., 2008). Bradshaw and colleagues (2012), for example, demonstrated significantly reduced rates of referrals for students in 21 campuses with SWPBIS intervention support compared to 16 campuses not receiving SWPBIS support. A practice encouraged within the OCR’s guidance (2012), “SWPBS provides a framework, like RtI [Response to Intervention], comprised of a behaviorally oriented conceptual foundation that has been enhanced by contributions from applied behavior analysis and positive behavior support” (Sugai & Horner, 2009, p. 234). Instead of focusing on punishment and negativity in disciplinary communications with students, SWPBS encourages school staff to proactively provide positive expectations for students and allow opportunities for students to self-correct undesired behaviors. Structured into a system of three Tiers, Tier 1 entails application 4 Global Education Review 9 (3) of PBIS concepts across all school settings. Tiers 2 and 3 more specifically focus on students who have not experienced success within the first tier (Sugai & Horner, 2009). Restorative Justice. At the time of the resolution agreement with the OCR, OUSD had already begun implementing SWPBS on select campuses to address discipline reform. In addition to SWPBS, the district had also begun the practice of restorative justice within specific schools. Restorative Justice (RJ), like SWPBS, encourages a positive approach to discipline and prioritizes students remaining in school. When a student commits an offense against another student on an RJ campus, a ‘talking circle’ is formed so that the offending student can hear the effect of his/her action and the circle can determine a resolution that restores harmony to the group and allows the offending student to feel reintegrated (Jain et al., 2014). In the agreement with the OCR, OUSD committed to continuing implementation of SWPBS and restorative justice on the identified campuses for intervention (referred to as Voluntary Resolution Plan [VRP] Cohort schools), though the agreement does not require a specific intervention or combination of interventions for the campuses (OCR, 2012). SWPBS, restorative justice, and other intervention programs for student discipline are not designed, however, to individually resolve all disciplinary reform needs without other techniques. By definition, SWPBS is an overall framework which incorporates multiple strategies under the umbrella of positive supports (Sugai & Horner, 2009) and includes rigorous attention to disaggregated discipline data as part of a continuing system of reviewing and modifying practices (Gregory et al., 2017; McIntosh et al., 2014). Thus, it takes several years to first establish the basic structure of positive expectations (McIntosh et al., 2016) and only then, through data disaggregation and analysis, can the campus SWPBS team determine the most appropriate supports needed at the campus. Additional interventions. Two added interventions have shown recent promise (Bradshaw, Pas et al., 2018; Cook et al., 2018). Double check coaching, a program in which teachers’ cultural sensitivity and communication skills are enhanced and encouraged through professional development and individualized coaching, resulted in significantly lower referrals for African American students (Bradshaw, Pas et al., 2018). The GREET-STOP-PROMPT (GST) technique, as tested by Cook and colleagues (2018), resulted in a reduced risk ratio of all students receiving referrals. Cook et al. (2018) further found that the remaining high-risk ratios for African American male students at one campus were due to a small set of teachers’ referrals and could be reduced with additional supports. In both studies, the researchers caution that the success of the interventions was inextricably linked to the existence of Tier 1 SWPBS practices on the campuses (Bradshaw, Pas et al., 2018; Cook et al., 2018). Theoretical Framework For this study, our interest in the district’s responses to OCR reviews in terms of changes in procedures, practices, and policies, leads us to consider the three phases of the policy reform process (i.e., policy talk, policy action, and implementation) as conceptualized by Tyack and Cuban (1995) and Peck and Reitzug (2012). These scholars suggest that policy talk describes the problem identification and temporary conversation surrounding the call for action; policy action describes the phase in which actual reform begins through school board policy change or legislation; and implementation refers to the more difficult phase of “putting reforms into practice” (Tyack Discipline Policy Reform 5 5 & Cuban, 1995, p. 40). Implementation may be compromised or may erode based on a variety of factors. For instance, Peck and Reitzug (2012), in their study of business management concepts implemented within school leadership, conceive an implementation cycle in which a leadership “fashion” declines and fades either as barriers to implementation arise or as the fashion goes out of style. Furthermore, successful policy implementation faces both internal and external challenges. Internally, educators may choose to comply with required reform yet “relegat[e] it to the periphery of the school” (Tyack & Cuban, 1995, p. 55). Externally, reforms have a greater likelihood of success when required by law and monitored for compliance (Tyack & Cuban, 1995). Tyack and Cuban (1995) alternatively argue against the concept of cycles of school reform in favor of a longer-term perspective in which the policy talk cycles, but implementation occurs over a lengthier timetable and often even after popular conversation regarding the issue has ceased. This study’s discussion will weigh the findings through the policy cycle perspective and will then consider Tyack and Cuban’s (1995) suggestion to view school reform over a larger span of time. Research Methods and Design For this study, we employed a descriptive case study design, which is appropriate in exploring a phenomenon (e.g., school discipline) within a specific context (i.e., OUSD), in order to provide a concise account of the facts and expert commentary to help in understanding the phenomenon, the forces behind the solutions, the outcomes of implementation, lessons learned, and connections to practice, policy, and theory (Gall et al., 2007; Yin, 2007). Also, critical to using this methodological approach is the flexibility in the types of data collected as “methodologists recommend that you take an eclectic approach and rely on any data that will help you understand your case and answer your research questions” (Johnson & Christensen, 2017, p. 436). Thus, the purpose of this current study was to examine OCR investigation of OUSD to determine discipline disparity of African American students and the consequent response by the district. The study was guided by three research questions: (1) In the OCR compliance review of Oakland USD concerning discipline disparities between African American and White students, what key elements of policy and practice did the OCR identify that needed reform? (2) What are the differences in discipline outcomes for African American students prior to and post OCR investigations of OUSD? (3) For each key element identified by the OCR, what stage of reform has OUSD achieved (policy talk, policy action, or implementation)? Data Sources and Collection Procedures Data were derived from OCR review findings, recommendations, and the resolution with Oakland Unified School District. We therefore considered OCR findings compared to the current state of discipline of African Americans in OUSD by assessing district discipline data. The OCR utilized both district data and data as reported to the Civil Rights Data Collection (CRDC) (ocrdata.ed.gov). We utilized the district data as reported to the OCR in their annual reports and which was extracted from OUSD’s data dashboards for incident numbers and the CRDC data for 2017 for numbers of students receiving infractions. Also, we examined accountability data to include student achievement data, which is a critical outcome factor that is descriptive of students’ educational experiences. In addition, district documents and external sources were searched 6 Global Education Review 9 (3) for information describing the initiatives that the district committed to within the Agreement to Resolve. District documents that were reviewed include Board documentation, district memos, department presentations, improvement plans, budgets, and meeting minutes and transcriptions. We also searched for media reports specifically referring to the OCR review for contextual reference and for reports of collaborative work done with outside organizations. Furthermore, we identified policy changes enacted in response to OCR recommendations as an effort to eliminate discipline disproportionalities. The document review included over 470 reports, action plans, board action items documentation, board meeting minutes, and board agendas. In addition, within each report filed with the OCR, OUSD included committee meeting minutes, agendas, training schedules, and data presentations for each category of activities required by the OCR. In the 2017 Annual Report to the OCR, for example, the answers to the OCR’s questions comprise 39 pages of data and an additional 372 pages of appendices. The appendices include meeting notes and transcriptions for 40 separate meetings of the Voluntary Resolution Plan Team (VRP Team) and related committees, survey results, action plans, and details for restorative justice and positive behavior trainings. Analysis Following initial data collection, we logged the largest collection of documents, the district records using a spreadsheet and categorized documents based on date, document title, author, money spent, and action taken as noted in the document. Next, we coded each document based on OCR’s identified key elements and created summaries for documents within each school year. The annual reports to the OCR were reviewed separately as discussed below. Key data elements and transcriptions were then extracted from the annual reports for further review. Following a full reading of transcriptions and meeting minutes, key quotes were extracted to represent the overall sentiment and/or key factual statements for each meeting. Rather than focusing on popular success stories of specific programs (e.g., see kingmakersofoakland.org for a docuseries regarding the AAMA), we instead pursued an exhaustive examination of each activity as identified within the OCR agreement (Wollmann, 2007). Using the full list of detailed activities within the OCR agreement (see Appendix A for further details) and the data collected, we determined the extent to which each activity was completed. Next, for each category of activity, we generated a rating using Peck and Reitzug’s (2012) stages of policy reform (policy talk, policy action, policy implementation). Next, we triangulated the gaps between policy implementation and the missing improvements in discipline practices through qualitative document analysis (Altheide et al., 2008) to consider if the meaning within the policy action and implementation matches the intent as identified by the OCR. Finally, we identified recurring and continuing challenges to implementation. Limitations Our decision to consider the full breadth of OUSD’s disciplinary practice reforms instead of targeting attention to one program (such as the AAMA) increases the complexity of the analysis. While the best practices potentially derived from a focus on one program could potentially benefit other districts, this study’s approach instead seeks to give a “comprehensive picture of what has happened in the policy field Discipline Policy Reform 7 7 and project under scrutiny, encompassing successful as well as unsuccessful courses of events...” (Wollmann, 2007, p. 394). Our approach was aided by the laudable availability of district documents through the OUSD website and the thorough nature of their reports to the OCR. Our attempts to contact district personnel for additional interviews, however, were unsuccessful. As the OCR reports demonstrate, the district spent a great deal of time creating a documentary record for future analysis. Our reliance on district documents limits the perspective of our analysis; however, our interest lies in how the programs, dialogue, and district perspectives changed over time and this information is represented within the document collection (Altheide et al., 2008). Findings The reporting of the findings is organized by research questions. RQ1: OCR identified that within the OUSD, African American students were disproportionately represented in exclusionary disciplinary actions e.g., out of school suspension. For instance, from 1999 to 2012, the risk ratio (RR) for African American students receiving out-of-school suspension (OSS) increased significantly i.e., from RR of 1.42 in 1998/99 to RR of 1.98 in 2011/12. Consequently, some of the key OCR recommendations included revising district discipline policies; expanding the number of schools (elementary, middle, and high) to implement programs to address discipline disproportionality; implementing Response to Intervention (RtI) framework; developing targeted support services for African American students; and hosting discipline policy discussion forums for parents. The total agreed activities, described in Appendix A, comprise a total of 160 line items, some of which list specific data points for the district to collect, disaggregate, and analyze; in addition to line items listing agreed targeted intervention activities. The agreed activities are listed within ten categories: (1) Collaboration with experts, (2) Preventative strategies, early identification of at-risk students, early intervention, (3) Revision of discipline policies, (4) Outreach to and input from stakeholders, (5) Training, (6) Informational programs for parents, (7) Notice to parents and students, (8) Discipline Review Committee: VRP Team, (9) Data Collection, evaluation, self-monitoring, and (10), School security officers. These steps ultimately became the set of recommended activities provided by the OCR in their “Dear Colleague” guidance to school districts regarding discipline disparity (Lhamon & Samuels, 2014). OUSD administration was aware that their work served as a model. In a meeting of the VRP Team with network superintendents on October 10, 2016, Barb noted, “I would add that they are really pressing us on fidelity to the models that we are lifting up. They said that we are the national model by which they are advising other districts” (Document 17-2185, Appendix 2, p. 75). RQ2: The total number of student suspensions decreased from 3,567 suspensions in 2012-13 to 1,936 suspensions in 2016-17. However, suspensions of African American students continue to indicate significant disparities. In 2011-12, African American students composed 31.8% of enrollment, yet composed 63% of students who were suspended and 61% of students who were expelled. In 2016- 17, African American students composed 26% of enrollment, yet composed 60% of students who were suspended and 53% of students who were expelled. Students receiving special education services fared even worse-they represented 32% of students receiving suspensions in 2016-17 and 64% of suspended students receiving special education services were African American. 8 Global Education Review 9 (3) District reports to the VRP Team and related committees further disaggregate discipline for VRP campuses. The total number of suspension incidents for African American students dropped from 2006 in 2012-13 to 917 in 2016-17, though the disparities remain. African American students received 60% of suspensions in the VRP schools despite representing only 28.7% of the population in VRP schools. RQ3: OUSD has taken most of the steps as outlined in the agreement with the OCR. These actions are documented in annual reports provided to the school board (the district committed to produce five annual reports) and OUSD has maintained transparency by publishing these reports online and hiring outside consultants to evaluate new programs. Table 1 identifies the stage of policy reform as defined by Peck and Reitzug (2012) for each main provision of the agreement with the OCR. Appendix B demonstrates a sample of the reforms and activities by year, as evidenced by documentation provided to the school board in 2012 and 2013. Despite the intensity of reform in OUSD over the last six years evidenced from the document review, three continuing challenges emerged following qualitative document analysis - data quality, training, and commitment to continued implementation. These challenges were identified both through data elements within the annual reports and they were also discussed in detail within OCR Meetings, VRP Leadership Meetings, and other associated committee meetings and trainings. The annual reports include transcripts of these meetings, providing context for our findings. Data quality. Prior to the OCR’s intervention, OUSD did not have a reliable system for accurately recording and tracking disciplinary incidents that was consistently applied at all campuses. As part of the required steps, OUSD created a Universal Referral Form (URF) and required its use throughout the district. This enormously complex process required an initial training for administrators and clerical staff, in addition to regular updates and training of new staff (an added challenge in the face of high turnover). With incomplete and/or inaccurate data, there is a question regarding if student discipline is accurately reflected and if it is appropriate to Table 1 Stages of Policy Reform for OUSD Discipline Reform Requirements Provision Stage (Talk/Action/or Implementation?) Notes Collaboration with experts Implementation Preventative strategies, early identification of at-risk students, early intervention Partial Implementation Strong internal and external support of RJ and AAMA a strength; Implementation fatigue and Admin support a concern re. PBIS; Not all VRP schools reached Implementation fidelity for Tier 1 Revision of discipline policies Implementation Outreach to and input from stakeholders Implementation Training Partial Implementation Training logs reflect incomplete numbers of staff received trainings; turnover also complicates training logistics Informational programs for parents Implementation Notice to parents and students Implementation Discipline Review Committee: VRP Team Implementation Continued success relies on this team maintaining strong administrative support Data Collection, evaluation, self- monitoring Partial Implementation Problems with data remain, though regular team meetings identify and work to resolve these issues School security officers Implementation Discipline Policy Reform 9 9 attribute success or failure of reforms to potentially bad data. As Sondra identified in the final year of OCR oversight in a VRP Leadership Meeting, We met with OCR on Friday. It came up from the legal perspective that our data will not be considered valid because of incomplete data. So we could focus on the beginning of the school year you are setting up systems. The teachers also point out the need for more consistency. How do you operationalize a goal like this? (Document 17-2185, Appendix 2, p. 14) Barb, in response to Sondra’s concern, replied, ...then there needs to be some accountability on the forms. Because the data really is terrible. We already have a list. I would like to share out Eddie Fergus’s analysis. It points out that we have wonderful practices but it’s implementation. (Document 17-2185, Appendix 2, p. 14) Several months later, in another VRP Team Meeting, Seth acknowledged, We obviously are at the tail end of the VRP. OCR signaled that they want us to have the data but also the systems to work that out. We want to take the next steps with building those systems and want to also have the capacity to recognize when the data are not representative of what’s really happening at the sites. (Document 17- 2185, Appendix 2, p. 76) Thus, while OUSD did not successfully achieve full implementation and accuracy with the URF, the development of the data analysis team represents a positive practice that enables continued improvement. Training. The fundamental shifts in practice for disciplinary approaches, added social supports, and new data systems resulted in an increased demand for precious staff time to participate in training sessions. Heavy staff turnover also created additional demand for new employees to learn the basic systems. As an administrator explained during a Safe and Strong Schools Committee Meeting, getting teachers into trainings is the single biggest barrier to implementing RJ: To be frank, getting RJ training to the teachers is hard. It’s the hardest thing to do. We have a huge calendar of trainings and the trainings are always full. But teachers and administrators are really hard to access. We have two training days for teachers every year. Usually both days are dedicated to instruction because we have so many first second and third year (new) teachers, and a lot of turnover. It’s a systemic problem. We would like to negotiate that every teacher gets this training, and trauma‐ informed training. Referrals come from new teachers – we have lots of emergency credential teachers. We need everybody trained in trauma‐informed de‐escalation. Every department wants to get in front of teachers, but there is not enough time to get them all of the training. We need coaching after training. (Document 17-2185, Appendix 2, p. 110) Commitment to continued implementation. Conversations within leadership teams illustrate a genuine interest in successfully implementing reform for the improvement of student outcomes mixed with a concern of what the district’s priorities will be following the intense OCR oversight. Toward the end of the oversight period, the VRP Team 10 Global Education Review 9 (3) continued work on both the data quality issue and the intensified efforts to provide cultural responsive training in order to address persistent implicit bias in staff when working with diverse students. In a VRP Team Meeting, when the team was discussing the issue of implicit bias, Jody stated, A lot of what we were trying to put down as SMARTE goals under addressing bias, we don’t even know what we are meaning with “culturally responsive.” I think it would be a great outcome for our “Oakland Way” to know our meaning of that. The journey is long in terms of being anti‐racist and changing implicit bias, but we can at least get clear on what we are talking about and begin to have culture change. (Document 17-2185, Appendix 2, p. 18) VRP Team members were aware that they needed to demonstrate progress. As Jean mentioned in a meeting with network superintendents, ...we may still have disproportionate data, but we need to show that we are doing everything in our power to eliminate bias and to change our practices so that they can see that whatever disproportionality is there is not a result of discrimination. (Document 17-2185, Appendix 2, p. 40) While it was part of the function of the VRP Team to interpret OCR requirements and guide the district through the process of reform, the question that many staff members asked towards the end was whether their reform efforts would continue once the OCR was no longer requiring regular accountability and updates of their activities. The district technically reached implementation phase for seven out of the ten key elements as identified by the OCR. By Tyack and Cuban’s (1995) definition, implementation is the slower practice of “putting reforms into practice” (p. 40). Though OUSD has fully or partially entered implementation for all agreed reforms, the continuing rates of disparity in disciplinary incidents indicate that the implementation process of reforms will require continuing support to achieve intended effects. Discussion Oakland USD committed to a broad range of activities in order to resolve the OCR’s investigation into their inequitable disciplinary practices. The district engaged in the activities as promised and all reforms are in (at least) a partial implementation stage. At this time, disproportionality remains in the suspension and expulsion rates for African American students. There are promising steps signaling that OUSD’s continued efforts may yet produce improved outcomes for all students in disciplinary practices. First, the VRP Team successfully initiated practices to review data, discuss, and modify programs as a result. The team initiated new steps in response to identified gaps, such as cultural responsiveness training. Indeed, there is growing evidence that culturally responsive teaching (Gay, 2010; Ladson-Billings, 1995; Larson et al., 2018) coupled with culturally responsive leadership (Khalifa, 2018; Khalifa et al., 2016; Merchant et al., 2013) have a significant influence on positive student outcomes. In addition to the newly established efficiencies in administration, review of meeting minutes reveals honest intent of administrators to question inequitable practices. However, one question that persists is whether the intent for the reduction in discipline disparity was solely driven by the OCR Discipline Policy Reform 11 11 agreement or if the district will continue to pursue change without OCR oversight. The challenges to disciplinary reform as identified by staff include data quality, training, and the aforementioned continued commitment to reform. A larger issue, only discussed in part within district documentation, rises in weighing the district’s PBIS implementation experience with best practices as identified throughout the research. OUSD has not reached fidelity of Tier 1 implementation on all campuses, nor even within the VRP schools. Campuses implementing PBIS with fidelity at Tier 1 within one year have a larger likelihood of overall success than campuses that take longer to implement PBIS (McIntosh et al., 2016). The mixture of interventions, programs, and practices employed by OUSD are not consistently tied to the recommended practice of first establishing Tier 1 PBIS supports, then selectively implementing targeted interventions based on emerging needs from data analysis and campus stakeholders (Gregory et al., 2017). As both Cook and colleagues (2018) and Gregory and colleagues (2017) caution, their successful results are from campuses in which PBIS Tier 1 supports have been implemented with fidelity. As the OCR oversight of OUSD ends, the potential for administrative support may waver in the face of other competing needs such as budgetary pressure for reductions. Ultimately, administrative support is a critical factor in maintaining campus application of PBIS (Andreou et al., 2015) and will determine the likelihood of continued work toward reducing discipline disparity in OUSD. Additionally, of concern is not only the continuing disciplinary outcome disparities within student ethnicities overall, but also the heightened impact of higher rates of exclusion for African American students receiving special education services. Students of color are overrepresented within special education; thus, they are particularly vulnerable to the negative impacts of suspension (Kramarczuk Voulgarides et al., 2017). Some scholars, in fact, question the assumption that special education itself is useful, finding that a student with multiple identities may suffer from the intersection of racism and ableism within a teacher’s practice (Cruz et al., 2021). Indeed, cultural training for teachers, when combined with PBIS implementation has been associated with a reduction of exclusionary discipline for overrepresented groups (Gregory et al., 2017). Particularly, scholars and teacher educators such as Gloria Ladson-Billings and Geneva Gay have, in their seminal works, long advocated for culturally responsive (or relevant) teaching (or pedagogy) (Gay, 2010; Ladson-Billings, 1994, 1995). These scholars collectively argue for a pedagogy that produces academic success, helps students develop positive ethnic and cultural identities, elevates students’ sensitivity to social inequalities, and enhances student-teacher relationships, classroom climate, and self- awareness. These core principles of culturally responsive pedagogy when applied in tandem have the potential to keep African American students engaged in the learning process, ultimately producing desirable educational outcomes (Hammond, 2015). OUSD’s experience with reforming its disciplinary practices illustrates the complexity of large-scale social reform. Tyack and Cuban (1995) caution against viewing reform as cyclical, explaining that the length of trends in school reform do not correspond with public attention. That is, reform will continue beyond the time that we discuss the changes. Previous examples of this phenomenon include the sharp increases in per-pupil expenditures in the 1920s and 1950s (during politically conservative decades) and the delayed curriculum development in the 1960s following the 1950s fear of falling behind the Soviet Union in math 12 Global Education Review 9 (3) and science. The curriculum development in the 1960s occurred despite the shift in public attention to social justice matters (Tyack & Cuban, 1995). Discipline reform, a massive undertaking as evidenced by this report, will likely reform over time, but the public will likely be focused on other issues by the time true reform is realized A longer-term perspective will enable us to consider if OUSD’s reforms are merely serving a compliance function or engaging social change. The OCR’s efforts contributed “symbolically important” policy talk that are further complicated by both the lengthy implementation process specific to behavioral system reform and the “uneven penetration of the reform[s]” as illustrated by uneven success on differing campuses (Tyack & Cuban, 1995, p. 55). After shifting away from legal monitoring for compliance during the Trump administration (Klein & Blad, 2018; Ujifusa, 2018), there is likelihood with the new Biden Administration’s apparent focus on social justice, to return once again to enforcing civil rights of learners to ensure equal access to education. Therefore, revisiting the state of the district, i.e., OUSD, in a couple of years will be necessary. Conclusions It is clear that much work remains in combating underlying contributing factors to disparate outcomes for African American students. OUSD’s annual progress reports to the OCR show continued progress in the number of schools implementing discipline reform, however training has remained optional in most cases, resulting in uneven implementation. Also challenging is the level of data accuracy and completion, as identified in OCR’s original report. The district now achieves 80% completion of referral forms at all campuses, thus earlier data did not include all incidents. In light of our findings, there are important considerations for OUSD. In an effort to provide equal educational opportunities for all students, it is critical to continually eliminate time away from the regular learning environment for African American students, especially African American students receiving special education services. Students missing valuable classroom time are deprived of needed skill development, resulting in an increased need for remediation and a greater likelihood of repeated behavioral incidents as a result (McIntosh et al., 2014). In addition to the efforts around implementing PBIS, Restorative Justice, and RtI, OUSD may need to consider making professional development mandatory such as culturally responsive pedagogy which provides an underlying rationale for these other initiatives. Also, given the positive effort OUSD has exerted in response to the OCR agreement, discipline disparities still exist; a thorough root cause analysis needs to be conducted to bring to surface the real issues. Furthermore, our findings raise other important research related questions worth exploring: As districts, at least initially, appear to put forth effort to respond to agreements with OCR, little impact on student outcomes is evident. This lack of improvement in outcomes for African American students begs a comprehensive investigation of the real root cause analysis of these student disparities; are OCR efforts really producing the change needed to educational equity? Finally, are the districts’ efforts merely for compliance or are they a commitment to true reform? Discipline Policy Reform 13 13 Appendix A Substantive Provisions within the Agreement to Resolve Oakland Unified School District OCR Case Number 09125001 Note: The following details provisions found within OUSD’s Agreement to Resolve with the OCR. For space considerations, the full 160 lines have been reduced. Instead, the main categories are listed here with a brief description for each, including a sample of the level of detail required by the agreement. The full list of agreed activities can be found at: https://www2.ed.gov/about/offices/list/ocr/d ocs/investigations/09125001-b.pdf I. Collaboration with experts Hire experts, appoint a project leader for the VRP team II. Preventative strategies, early ident of at- risk students, early intervention Select program for VRP schools that utilizes equitable and restorative discipline practices; evaluate program effectiveness; build a trauma team; implement recommendations of AAMA Task Force III. Revision of Disciplinary Policies Review data and consider policy revisions; align Board policy with district initiatives IV. Outreach to and Input from Stakeholders Provide reports to OCR; administer surveys; conduct forums for students and parents V. Training Provide annual staff and student training for VRP schools that emphasizes PBIS VI. Informational Programs for Parents Develop parent program that informs on due process, positive behavior, and district contact information VII. Notice to Parents and Students Revise parent and student handbooks; provide notice of Ombudsman; inform public of discipline data VIII. Discipline Review Committee: VRP Team Establish VRP Team which will then develop frameworks, systems, targeted reductions, training schedules IX. Data Collection, Evaluation, Self- Monitoring Add tracking of site based discipline and teacher initiated suspension; accurately collect discipline data; convene software user forum; provide database training; produce and share discipline reports; district and campus meetings will consider discipline data X. School Security Officers Review SSO program to determine if it aligns to intent of this agreement; conduct training and policy changes as needed for alignment 14 Global Education Review 9 (3) Appendix B Sample of Document Review - Documents presented to the School Board 2012-2013 (full review includes 2012-2017) Document Date Document Title Summary of Document Agreement to Resolve Correlations March 27, 2013 Professional Services Contract Contract between OUSD and USC for design support and analysis of findings of OUSD 0-8 AAMA for 2012-2013. I.a. - consult w experts in data analysis and research-based strategies on AA and school discipline III.c. - Phase I - district reviews its 2010-2012 discipline data IX.a. - District will gather and review electronic data. May 8, 2013 Professional Services Contract Contract between Oakland Tech HS and Mentoring Center for mentoring training. V.a. – VRP Cohort school training (includes programs, training with connections to fairness and non- discrimination based on race/ethnicity. V.b. - District will develop VRP Cohort School staff training module May 8, 2013 District Submitting Grant Application Grant application to support staffing costs for AAMA Initiative. II.e. - AAMA task force recommendations (mentoring, t/s relationships, site mentoring programs, timeframe for monitoring and evaluation) May 8, 2013 Professional Services Contract Amendment Approval of Partners in School Innovation for Manhood development programs and outreach to AA families. I.a. - consult w experts in data analysis and research-based strategies on AA and school discipline Discipline Policy Reform 15 15 References Algozzine, B., Horner, R. H., Sugai, G., Barrett, S., Dickey, S. R., Eber, L., Kincaid, D., et al. (2010). Evaluation blueprint for school-wide positive behavior support. National Technical Assistance Center on Positive Behavior Interventions and Support. Retrieved from www.pbis.org Altheide, D., Coyle, M., DeVriese, K., & Schneider, C. (2008). Emergent qualitative document analysis. In S. N. Hesse-Biber & P. Leavy (Eds.), Handbook of Emergent Methods (pp.127-151). Guilford Press. Andreou, T. E., McIntosh, K., Ross, S. W., & Kahn, J. D. (2015). Critical incidents in sustaining school-wide positive behavioral interventions and supports. The Journal of Special Education, 49(3), 157-167. Arcia, E. (2006). Achievement and enrollment status of suspended students: Outcomes in a large, multicultural school district. Education and Urban Society, 38(3), 359-369. Authors (2018). Bradshaw, C. P., Pas, E. T., Bottiani, J. H., Debnam, K. J., Reinke, W. M., Herman, K. C., & Rosenberg, M. S. (2018). Promoting cultural responsivity and student engagement through Double Check coaching of classroom teachers: An efficacy study. School Psychology Review, 47(2), 118-134. Bradshaw, C. P., Koth, C. W., Bevans, K. B., Ialongo, N., & Leaf, P. J. (2008). The Impact of School-Wide Positive Behavioral Interventions and Supports (PBIS) on the Organizational Health of Elementary Schools. School Psychology Quarterly, 23(4), 462–473 Bradshaw, C. P., Waasdorp, T. E., & Leaf, P. J. (2012). Effects of school-wide positive behavioral interventions and supports on child behavior problems. Pediatrics, 130(5), 2012- 0243. Civil Rights Act of 1964. Pub.L. 88–352, Title VI, Sec. 601, 78 Stat. 252 (1964). Cook, C. R., Duong, M. T., McIntosh, K., Fiat, A. E., Larson, M., Pullmann, M. D., & McGinnis, J. (2018). Addressing discipline disparities for Black male students: Linking malleable root causes to feasible and effective practices. School Psychology Review, 47(2), 135-152. Cruz, R. A., Kulkarni, S. S., Firestone, A. R. (2021). A quantcrit analysis of context, discipline, special education, and disproportionality. AERA Open. https://doi.org/10.1177/23328584211041354 D’Orio, W. (2018, January). Is school-discipline reform moving too fast? The Atlantic. Retrieved From https://www.theatlantic.com/education/archive /2018/01/is-school-discipline-reform-moving- too-fast/550196/ Gall, M. D., Gall, J. P., & Borg, W. R. (2007). Educational research: An introduction (8th ed). Pearson. Gay, G. (2010). Culturally responsive teaching: Theory, research, and practice (2nd ed.). Teacher College Press. Gregory, A., & Fergus, E. (2017). Social and emotional learning and equity in school discipline. The Future of Children, 117-136. Gregory, A., Skiba, R. J., & Mediratta, K. (2017). Eliminating disparities in school discipline: A http://www.pbis.org/ http://www.pbis.org/ http://www.pbis.org/ https://doi.org/10.1177/23328584211041354 https://www.theatlantic.com/education/archive/2018/01/is-school-discipline-reform-moving-too-fast/550196/ https://www.theatlantic.com/education/archive/2018/01/is-school-discipline-reform-moving-too-fast/550196/ https://www.theatlantic.com/education/archive/2018/01/is-school-discipline-reform-moving-too-fast/550196/ 16 Global Education Review 9 (3) framework for intervention. Review of Research in Education, 41(1), 253-278. Gregory, A. & Weinstein, S. R. (2008). The discipline gap and African Americans: Defiance or cooperation in the high school classroom. Journal of School Psychology, 46, 45-475. Hammond, Z. (2015). Culturally responsive teaching and the brain: Promoting authentic engagement and rigor among culturally and linguistically diverse students. Corwin. Jain, S., Bassey, H., Brown, M. A., & Kalra, P. (2014). Restorative justice in Oakland schools: Implementation and impact. Data In Action, LLC. Retrieved from http://www.ousd.org/cms/lib07/CA01001176/C entricity/Domain/134/OUSDRJ%20Reprt%20r evised%20Final.pdf Johnson, R. B. & Christensen, L. (2017). Educational research: Quantitative, qualitative, and mixed approaches (6th ed.). Sage. Khalifa, M. (2018). Culturally responsive school leadership. Harvard Education Press. Khalifa, M. A., Gooden, M. A., & Davis, J. E. (2016). Culturally responsive school leadership: A synthesis of the literature. Review of Educational Research, 86(4), 1272-1311. Klein, A. & Blad, E. (2018). Scrap discipline guidance, consider arming school staff, Trump commission says. Education Week. Retrieved from https://www.edweek.org/ew/articles/2018/12/1 8/scrap-discipline-guidance-consider-arming- school-staff.html?print=1 Kozol, J. (1991). Savage inequalities. Crown Publishers. Kramarczuk Voulgarides, C., Fergus, E., & King Thorius, K. A. (2017). Pursuing equity: Disproportionality in special education and the reframing of technical solutions to address systemic inequities. Review of Research in Education, 41(1), 61-87. https://doi.org/10.3102/0091732X16686947 Ladson-Billings, G. (1998). Just what is critical race theory and what's it doing in a nice field like education? International Journal of Qualitative Studies in Education, 11(1), 7-24. Ladson-Billings, G. (1995). But that’s just good teaching! The case for culturally relevant pedagogy. American Educational Research Journal, 32, 159–165. Ladson-Billings, G. (1994). The dreamkeepers: Successful teachers of Black children. Jossey- Bass. Larson, K. E., Pas, E. T., Bradshaw, C. P., Rosenberg, M. S., & Day-Vines, N. L. (2018). Examining how proactive management and culturally responsive teaching relate to student behavior: Implications for measurement and practice. School Psychology Review, 47(2), 153-166. Lhamon, C.E. (2014, October 1). Dear colleague letter: Resource comparability. U.S. Department of Education’s Office for Civil Rights. Retrieved from https://www2.ed.gov/about/offices/list/ocr/lett ers/colleague-resourcecomp-201410.pdf Losen, D. J. & Gillespie, J. (2012). Opportunities suspended: The disparate impact of disciplinary exclusion from school. The Center for Civil http://www.ousd.org/cms/lib07/CA01001176/Centricity/Domain/134/OUSDRJ%20Reprt%20revised%20Final.pdf http://www.ousd.org/cms/lib07/CA01001176/Centricity/Domain/134/OUSDRJ%20Reprt%20revised%20Final.pdf http://www.ousd.org/cms/lib07/CA01001176/Centricity/Domain/134/OUSDRJ%20Reprt%20revised%20Final.pdf https://www.edweek.org/ew/articles/2018/12/18/scrap-discipline-guidance-consider-arming-school-staff.html?print=1 https://www.edweek.org/ew/articles/2018/12/18/scrap-discipline-guidance-consider-arming-school-staff.html?print=1 https://www.edweek.org/ew/articles/2018/12/18/scrap-discipline-guidance-consider-arming-school-staff.html?print=1 https://doi.org/10.3102/0091732X16686947 https://www2.ed.gov/about/offices/list/ocr/letters/colleague-resourcecomp-201410.pdf https://www2.ed.gov/about/offices/list/ocr/letters/colleague-resourcecomp-201410.pdf Discipline Policy Reform 17 17 Rights Remedies at the Civil Rights Project: University of California at Los Angeles. Maxwell, L. A. (2011). Civil rights office expands its reach into schools. Education Week. 31(14), 1. McIntosh, K., Mercer, S. H., Nese, R. N., Strickland-Cohen, M. K., & Hoselton, R. (2016). Predictors of sustained implementation of school-wide positive behavioral interventions and supports. Journal of Positive Behavior Interventions, 18(4), 209-218. McIntosh, K., Ty, S. V., & Miller, L. D. (2014). Effects of school-wide positive behavioral interventions and supports on internalizing problems: Current evidence and future directions. Journal of Positive Behavior Interventions, 16(4), 209-218. Merchant, B., Garza, E., & Ramalho, E. M. (2013). Culturally-responsive leadership. In C. Day & D. Gurr (Eds.), Leading schools successfully: Stories from the field (pp. 174– 183). Routledge. Okonofua, J. A., & Eberhardt, J. L. (2015). Two strikes: Race and the disciplining of young students. Psychological Science, 26(5), 617-624. doi:10.1177/0956797615570365 Office for Civil Rights, U.S. Department of Education. (2012). Title VI enforcement highlights. Retrieved from https://www2.ed.gov/documents/press- releases/title-vi-enforcement.pdf OCR, USDEO (2012). Compliance review: Oakland Unified School District, Oakland. Retrieved from https://www2.ed.gov/about/offices/list/ocr/doc s/investigations/09125001-a.html Okilwa, N. S., Khalifa, M., & Briscoe, F. (Eds.) (2017). The school to prison pipeline: The role of culture & discipline in school. Emerald Publishing Ltd. Peck, C., & Reitzug, U. C. (2012). How existing business management concepts become school leadership fashions. Educational Administration Quarterly, 48(2), 347-381. Pernell, B. D. (2015). Aligning "educational necessity" with Title VI: An enhanced regulatory role for executive agencies in Title VI disparate impact enforcement. New York University Law Review, 90(4), 1369. Skiba, R. J., Horner, R. H., Chung, C-G., Rausch, M. K., May, S. L., & Tobin, T. (2011). Race is not neutral: A national investigation of African American and Latino disproportionality in school discipline. School Psychology Review, 40(1), 85-107. Skiba, R. J., Michael, R. S., Nardo, A. C., & Peterson, R. (2002). The color of discipline: Sources of racial and gender disproportionality in school punishment. Urban Review, 34, 317–342. Smolkowski, K., Girvan, E. J., McIntosh, K., Nese, R. N., & Horner, R. H. (2016). Vulnerable decision points for disproportionate office discipline referrals: Comparisons of discipline for African American and White elementary school students. Behavioral Disorders, 41(4), 178-195. Sugai, G., & Horner, R. H. (2009). Responsiveness-to-intervention and school-wide https://www2.ed.gov/documents/press-releases/title-vi-enforcement.pdf https://www2.ed.gov/documents/press-releases/title-vi-enforcement.pdf https://www2.ed.gov/about/offices/list/ocr/docs/investigations/09125001-a.html https://www2.ed.gov/about/offices/list/ocr/docs/investigations/09125001-a.html 18 Global Education Review 9 (3) positive behavior supports: Integration of multi- tiered system approaches. Exceptionality, 17(4), 223-237. Tyack, D. B., & Cuban, L. (1995). Tinkering toward utopia. Harvard University Press. Ujifusa, A. (2018). Betsy DeVos revokes Obama discipline guidance designed to protect students of color. Education Week’s Blog. Retrieved from http://blogs.edweek.org/edweek/campaign-k- 12/2018/12/betsy_devos_revokes_obama_disci pline_guidance_students_of_color_protect.htm l USDOE, OCR (2018). Overview of the agency. Retrieved from https://www2.ed.gov/about/offices/list/ocr/abo utocr.html Wollmann, H. (2007). Policy evaluation and evaluation research. In F. Fischer, G. Miller, & M. Sidney (Eds.), Handbook of Public Policy Analysis (pp. 393-402). Taylor & Francis. Yin, R. K. (2009). Case study research: Design and methods (4th ed.). Sage. http://blogs.edweek.org/edweek/campaign-k-12/2018/12/betsy_devos_revokes_obama_discipline_guidance_students_of_color_protect.html http://blogs.edweek.org/edweek/campaign-k-12/2018/12/betsy_devos_revokes_obama_discipline_guidance_students_of_color_protect.html http://blogs.edweek.org/edweek/campaign-k-12/2018/12/betsy_devos_revokes_obama_discipline_guidance_students_of_color_protect.html http://blogs.edweek.org/edweek/campaign-k-12/2018/12/betsy_devos_revokes_obama_discipline_guidance_students_of_color_protect.html https://www2.ed.gov/about/offices/list/ocr/aboutocr.html https://www2.ed.gov/about/offices/list/ocr/aboutocr.html