Impacting Education: Journal on Transforming Professional Practice New articles in this journal are licensed under a Creative Commons Attribution 4.0 United States License. This journal is published by Pitt Open Library Publishing. 63 This journal is supported by the Carnegie Project on the Education Doctorate: A Knowledge Forum on the EdD (CPED) cpedinitiative.org impactinged.pitt.edu ISSN 2472-5889 (online) Vol. 9 No. 1 (2024) DOI 10.5195/ie.2024.386 (Re)designing a CPED-Oriented EdD Program to Improve its Emphasis on Equity in a Post Pandemic World Henry Tran University of South Carolina htr@sc.edu Kathleen Cunningham University of South Carolina katiemwc@mailbox.sc.edu Suzy Hardie University of South Carolina sehardie@mailbox.sc.edu Peter Moyi University of South Carolina moyi@mailbox.sc.edu Era Roberts School District of Newberry County era@dreraroberts.com ABSTRACT This paper focuses on the Education Systems Improvement EdD program (EDSI) at the University of South Carolina and how the program faculty utilize the signature methodology of the program, improvement science, towards its improvement efforts towards enhancing its equity focus, especially post-pandemic. We utilize the framework of improvement science including tools such as gap analysis, program evaluations, empathy interviews, and focus groups to better understand the problem and best design the appropriate improvement efforts. Broadly, three program improvement foci were identified for the improvement arc: program purpose, curricular design, and continuous improvement processes. Findings from this study provide details about program improvement efforts in improving an EdD program’s equity focus post-pandemic, in a particular setting. In the spirit of continuous improvement, this work is never done. It will be forwarded by follow-up improvement endeavors and activities. KEYWORDS continuous improvement, Education Doctorate, program improvement and redesign, diversity, equity, inclusion INTRODUCTION Equity and diversity represent core values of the Education Systems Improvement (EDSI) Doctorate of Education (EdD) program at the University of South Carolina (USC). Concurrently, our EdD is guided by the Carnegie Project on the Education Doctorate (CPED), whose values include diversity, learning, partnership, people, social justice, and a student-centered disposition to inform a future where equity-minded educational professionals lead toward sustainable change for the benefit of local communities. Several high profile racial and gender-based incidents empowered numerous social movements towards this end, including Black Lives Matter, Stop Asian Hate and #Metoo (Greene et al., 2019; Kaskazi, 2021). Institutions across most sectors responded with statements of solidarity with and/or commitments to improve equity-related issues in their respective fields, with education at the forefront given its influence on life opportunities for students. The pandemic exacerbated many of the issues of inequity. Among the major consequences of the COVID-19 pandemic was the exposing of deeply entrenched inequities embedded in our systems and structures—evidenced by the disproportionately poor outcomes for people of color in areas such as mental/physical health (Bambra et al., 2020; Gibson et al., 2021; Krouse, 2020), and education (e.g., lack of Wifi for remote learning) (Goudeau et al., 2021; Reimers, 2022). While change for improving equity was pressing before, it has become critical, especially for future education leaders. CPED- oriented EdD doctoral programs are designed to develop leaders with the dispositions and skills to evaluate and improve the systems and programs in which they work to respond to this change to be more just and equitable instead of fixing the individuals within them. This paper focuses on the EDSI EdD program at the University https://library.pitt.edu/e-journals http://creativecommons.org/licenses/by-nc-nd/3.0/us/ http://cpedinitiative.org/ Tran et al. Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 9 No. 1 (2024) DOI 10.5195/ie.2024.386 64 of South Carolina and how the program faculty utilize the signature methodology of the program, improvement science, towards its improvement efforts towards enhancing its equity focus, especially post-pandemic. While there have already been existing efforts developed to meaningfully integrate diversity, equity, and inclusion (DEI) as part of the DNA of this program, this article captures part of the program’s efforts to engage in continuous program improvement efforts to strengthen its emphasis on DEI post-pandemic. Knowing that they wanted to make improvements, the EdD faculty applied for and was awarded an internal grant that supported some of the program improvement work. This article captures improvement efforts related to improvement efforts writ large, some of which were connected directly to the internal grant, while other improvement efforts took place external to the grant work. This paper captures one period of time reflecting the program’s current understanding and work progress; indeed, in the spirit of continuous improvement, the work is not done and will continue to evolve. PROGRAM BACKGROUND The EdD in Education Systems Improvement (EDSI) is one of four strands of a College of Education-wide EdD degree at USC. At the time of this paper’s development, the program has existed for only three years and therefore still in its beginning stages. Housed in the College of Education’s Educational Leadership and Policy Department (EDLP), the EDSI EdD program emphasizes an improvement science framework as its signature pedagogy and methodology. This approach positions leaders who are educators, advocates, and improvers to address complex, adaptive problems of inequity by deeply understanding a problem and the system that is producing that problem, and then using different strategies to meaningfully disrupt educational systems of inequality to develop positive impactful changes (Hinnant-Crawford, 2020). At the time of EDSI program approval, the stated EdD program outcomes were: 1) Develop scholar practitioners with a strong foundation and strategies for education systems improvement, 2) Develop scholar practitioners with advanced understandings of district, state, and national policies, 3) Develop scholar practitioners to use principles of improvement science to solve systemic problems of practice in their contexts, and 4) Develop transformational leaders to positively impact student achievement, student engagement, school climate, students’ overall educational experience, and social change. EdD faculty are committed to modifying these original outcomes for depth and articulation to more directly and impactfully align with the College of Education’s priorities and CPED’s vision of “...a future where equity-minded educational professionals lead lasting and positive change for the learning and benefit of everyone” (CPED, 2021, para 2). While other educational leadership preparation programs in the department have articulated leadership dispositions (i.e., integrity, intellectual spirit, social justice, and initiative) that are included in course syllabi and aspiring leaders’ reflections and evaluation materials, the EDSI EdD is in the process of working to formally adopt its own. The EDSI EdD program employs a diverse, and knowledgeable group of approximately twelve full-time faculty members with additional educational advocates who serve as adjunct faculty. Together, they promote interdisciplinary expertise, intellectual spirit, and scholarly excellence for all learners. Students are organized into cohorts of 20-25 students, who are adult learners and work full time across P-20 contexts. The program offers 100% online courses on topics such as, but not limited to, anti-racist and transformational leadership, improvement science, data-informed decision making, research methods, educational policy, and organizational theory and systems design. Students further participate in a two-part improvement science institute after semesters two and five, and take additional core courses in two other departments across the College of Education. EDD PROGRAM REDESIGN The EDSI EdD program emphasizes continuous improvement as described by Park and colleagues (2013), who define it as frequent or part of regular practice, focused on positive changes in systems and the multiple levels within those systems, including the processes that exist within those different levels of the system. Coupled with a continuous improvement orientation, faculty expertise in areas such as educational systems, leadership, policies, and improvement set the stage for identifying and addressing opportunities for improving equity in the field of education post- pandemic. The commitment to continuous improvement of the program’s systems and processes invited the opportunity to focus on a specific area of the redesign effort: advancing the EdD’s articulated emphasis on equity. The improvement team used various data sources to identify places and ways to improve. The multi-pronged approach to the redesign efforts included: (a) a syllabi review and gap analysis, (b) student feedback, (c) program evaluations, (d) faculty feedback, (e) empathy interviews and discussions, (f) literature, research, and examples from analogous settings, as well as (g) a root cause analysis diagram (i.e., fishbone diagram). These data were considered at different points of the redesign efforts. We will describe a portion of the EDSI improvement efforts and actions in three phases: Phase 1 includes the initial improvements made external to the grant-related efforts, Phase 2 includes the improvement processes related to the grant-related efforts, and Phase 3 offers future improvement efforts and actions beyond the grant. Phase 1: Initial Program Improvements Under the leadership of the EDSI EdD program coordinator and the department chair, the first major component of the efforts to improve the program’s emphasis on equity and diversified perspectives included introducing new faculty and courses. For example, two new clinical faculty members with expertise in higher education (which was under-represented in the curriculum despite the program being a P-20 focus) and equity were hired to teach in the program. In addition, new courses focusing on inclusion were developed for students in the program during the improvement efforts. The first of these courses (i.e., Inclusive Talent Management in Education - Cultivating a Work Climate for Inclusive Excellence) will be offered next summer, with companion courses offered the year after. Phase 2: Grant-Related Improvement Efforts The second major component of the program improvement efforts is led by a team that includes the EdD program coordinator, the department chair, the MEd in K-12 leadership’s program coordinator, an assistant professor who teaches the improvement science courses, a current EdD student, and a practicing educational Improving Equity Focus in an EdD Program Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 9 No. 1 (2024) DOI 10.5195/ie.2024.386 65 leader who serves as an adjunct professor in the MEd program with expertise in program improvement efforts. This team anchored their understanding of the improvement area (i.e., advancing the EdD emphasis on equity) into three areas: (1) program purpose, (2) curricular design, and (3) continuous improvement processes. The next portions of this paper will illustrate the team’s approach to the (re)design process and draw on existing research in improvement science (e.g., Anderson et al., in press-a; Bryk et al., 2015; Hinnant- Crawford, 2020; Langley et al., 2012) and in program improvement and redesign (e.g. Young et al., 2022) to anchor these efforts. The Approach to Improvement Improvement efforts were guided by improvement science dispositions and mindsets (e.g., Anderson et al., 2023; Biag & Sherer, 2021), while drawing on some (not all) improvement science methods taught in the EdD program. Biag and Sherer (2021) offer that improvers embody particular dispositions such as commitment to a learning mindset and a system’s perspective, willingness to take action for improvement, dedication to seeking additional perspectives, and perseverance in their priority to continuous improvement. These dispositions coordinate with the six principles of improvement as introduced by Bryk and colleagues (2015):  Make the work problem-specific and user-centered  Focus on variation in performance  See the system that produces the current outcomes  Measurement is necessary for scaling improvement efforts  Disciplined inquiry drives improvement  Networked communities can accelerate learning While the full improvement science methodology was not employed entirely, two of these principles– making the problem-specific and user-centered, as well as seeing the system that produces the current outcomes were and remain particularly helpful principles for the EDSI EdD improvement team. Improvement science invites improvers to spend time deeply understanding the problem they want to address. Strategies such as examining local qualitative and quantitative data, scanning outside the organization, and engaging in empathy interviews facilitate increased understanding (e.g., Carnegie Foundation for the Advancement of Teaching, 2018; Hinnant-Crawford, 2021; Nelsestuen & Smith, 2020). We describe how each facilitated the group’s continuous improvement processes. The multi-pronged program design approach involved the examination and analysis of the program’s artifacts (e.g., syllabi review), soliciting feedback to determine the current state of the program (e.g., empathy interviews, guided interview focus group with faculty, student-led program evaluations), and arranging information into a fishbone diagram to better understand the factors impacting the improvement focus. Understanding the Current Improvement Area: Program Artifacts (i.e., Program Outcomes, EDLP Course Information). Informational artifacts (i.e., program outcomes) helped the team orient the curriculum to the program outcomes and dispositions. The process involved collecting multiple data sources including input from current EdD students, course syllabi and seeking course information through clarifying interviews with instructors to capture details related to course design, topics, and enacted learning experiences. To identify specific directions for program and curricular improvement, the team needed to get a clear sense of the program’s state. Determining whether students are studying the desired curriculum is of utmost importance. Students are expected to learn; however, if they are not studying a curriculum aligned with program goals and values, their mastery will have a limited impact on equity. One way to determine this was by reviewing course syllabi to conduct a gap analysis process. Guided by a review template developed by Bryan (2021) at the Office of Diversity, Equity, & Inclusion at USC, the research team completed a review and gap analysis on the existing department offered courses (i.e., EDLP courses) in the EdD Program. The analysis involved two distinct but related phases. First, the program goals and outcomes were revisited. Then, additional program data sources (e.g., syllabi, faculty insights) were collected and reviewed to see how the sources aligned or diverged from the program outcomes. A key focus was to assess to what degree DEI topics were currently woven throughout the EDSI courses to represent a values-based cohesive core curriculum framed around equity. Thus, the focus of the gap analysis was to systematically reveal how DEI content was embedded in the readings, instructional activities, and the assessments. The MEd program coordinator facilitated the preparation and procedures for an associate professor, an EdD student, and a practicing school leader for analyzing seven course syllabi from the EDLP EdD program. The MEd program coordinator facilitated an initial preliminary review. Then, a more detailed review using the template (Bryan, 2021) took place. During this stage, the depth of knowledge (DOK) level definitions were used (Webb, 2002) to assess the DOK level of DEI (a) topics and (b) assessments in each EDLP course: introductory (I) indicated the presence of knowledge or skill where students are, for example, asked to recall or explain facts or concepts, advanced (A) knowledge or skill signaled that students were invited to apply a procedure or analyze how parts relate to or contrast from one another, or mastery (M) required students to evaluate or make judgments based on criteria or create a novel approach, product, or artifact. Each team member was randomly assigned syllabi to assess and categorize based on the DOK levels. Their reviews were collected on the template. Syllabi Review and Gap Analysis Results Seven education leadership and policies courses in EDSI were reviewed. From the preliminary review, the team made important observations that could inspire improvement directions. First, evidence suggested a lack of clarity around the program’s foundations or course details. Specifically, the overarching program outcomes were not fully articulated, included, or operationalized, which signaled that these were unclear to syllabi/course developers. Program outcome one states that the EdD program will develop scholar practitioners with a strong foundation and strategies for education systems improvement. The gap analysis revealed that the students are expected to analyze and interpret the theory of organizational power, develop a transformative equity action plan, use improvement science tools like plan, do, study, act (PDSA) cycles and empathy interviews to disrupt inequities and articulate ethical and social justice dimensions of educational policies and practices. EDSI EdD students are expected to develop a research equity action plan, develop a project proposal, write a code of ethics, and develop a client task report based on available organizational data. Instructional activities aligned with this program outcome included weekly readings and reflection, podcasts, and discussion posts. Tran et al. Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 9 No. 1 (2024) DOI 10.5195/ie.2024.386 66 Program outcome two states that the EDSI EdD will develop scholar practitioners with advanced understandings of district, state, and national policies. There was limited direct evidence across the syllabi (outside of the Educational Policy course) that students are developing advanced knowledge of district, state, and national policies. Although it could be argued that the readings provide an understanding of the policy context and the research equity action plan, it is important to note that project proposal, code of ethics, and client task report can only be developed with an understanding of the policy context. Program outcome three states that the EDSI EdD program will develop scholar practitioners to use principles of improvement science to solve systemic problems of practice in their contexts. The EDSI program is centered on the principles of improvement science; hence, students take an improvement science methodology course and two summer improvement science institutes to develop their skills to address problems of practice in their contexts. Students are also expected to create a data plan calendar and a data profile. Program outcome four states that the EdD program will develop transformative leaders to positively impact student achievement, student engagement, school climate, the overall educational experience, and social change. Syllabi content suggests that courses provide examples of anti-racist and transformative leadership, further developing students’ improvement science skills. Second, the information contained in some syllabi were incomplete (e.g., assignments, readings were not included). The gap analysis focused on to what extent DEI topics (such as racism; classism; heterosexism, cisheterosexism, homophobia; sexism and patriarchy; anti-Indianism; xenophobia; anti-semitism; bilingualism; and first-generation Americans) were embedded in the program’s core courses’ readings, instructional activities, and assessments. Since there was a deficit of all aspects of DEI revealed, this information was presented to all EDLP professors so they could analyze their courses to incorporate understandings of the policies. This could be the first step for professors to revise their syllabi to more purposefully incorporate DEI’s comprehensive definition which could invite course design to reflect DEI, and incorporate intentional diversity in their course readings, class discussions, and assignments. We make two assumptions about the syllabi: 1. that the syllabus is a tool that serves as an agreement between the instructor and the student about the expectations and content of the course and 2, we assume that what is explicitly included in a course syllabus is limited because syllabi are unlikely to exhaustively include all course information and fully articulate what is missing within the documents. In other words, an accurate depiction of the totality of what is happening in coursework and the class outside of what can be captured in syllabi was limited. These assumptions connect to our improvement purpose which was to use available artifacts and information (e.g., empathy interviews as described later in this paper) to help identify future directions for improvement and share those with the program’s core instructors. It was important to supplement the findings of the gap analysis since syllabi cannot capture everything that is covered in a course. Therefore, beyond reviewing the syllabus documents, one team member facilitated a meeting with two other full-time faculty members who teach EDSI EdD courses to learn and discuss more about what is included in the course beyond what is outlined in the syllabi. Information from this group revealed that there were indeed deliberately designed discussions within course meetings that threaded DEI priorities into them. For instance, in the data informed decision-making course, students engage in a learning activity that requires them to review data from different example cases. In their reviews, students uncover inequities in the cases (e.g., trends in NAEP scores, underrepresentation of women in STEM). In a similar way, in the economics of education course, the instructor facilitates multiple entry points for students to grapple with how current systems advantage and disadvantage people and how systematic and systemic processes do not allow certain populations to be able to enter pipelines of opportunities. Further, this instructor highlighted how facilitating the personalization of the topics invites students to reflect on how course themes impact them and their leadership. This additional information illustrates why a multi-faceted assessment is needed when aiming to improve a program or a component of the program. A more holistic understanding of courses invites more precision when working to improve upon existing learning activities. Understanding the Improvement Area: Student-Led Program Evaluations The improvement efforts for the program are informed and forwarded by annual internal program evaluations. Specifically, through their coursework, EDSI EdD students engage in authentic learning through conducting a program evaluation on their own EDSI EdD program. The program evaluation instrument is the result of an adaptation and revalidation of Kim and colleague’s (2020) Quality of Nursing Doctoral Education (QNDE) instrument for the EdD context, framed with the CPED’s (2021) guiding principles of improvement, and focus group feedback from students. The data are collected from students in the program and give program faculty a benchmark of the program effectiveness, identification of program strengths to continue to emphasize, and refine areas for improvement. Their perspectives are uniquely important because EdD students were working and in EdD coursework during and following the COVID-19 shutdown. Timely information is critical for facilitating effective responses to feedback. Thus far, data from two cohorts have been collected and analyzed. The surveys were distributed online via the Qualtrics platform and sent to the email addresses of each student in particular program cohorts. There was no time limit allotted for completion nor were there any incentives for completion other than the ability to provide feedback to improve the program. The first-year cohort questionnaire had 31 questions spread across five domains (one of which is focused on social justice and equity) and three open ended response questions. The survey for cohort one yielded a response rate of 59%, with a strong Chronbach alpha (α=.98), and scale level content validity index (S-CVI = .97) based on agreement levels of a panel of experts. For cohort two, the questionnaire was revised and encompassed 30 questions spread across four domains (the domain of honoring community and context was removed from this iteration of the survey), with three open-ended response questions. The questionnaire for cohort two yielded a response rate of 78%, with strong internal reliability (α=.83) and content validity (S- CVI = .91). Results from the data are shared with the cohort respondents and additional recommendations for improvement are also solicited at this time. Those results and recommendations are then shared with the EDSI EdD faculty in the department’s annual fall retreat. The faculty discuss the quantitative and qualitative data, and ideas for action plans for each recommendation are generated Improving Equity Focus in an EdD Program Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 9 No. 1 (2024) DOI 10.5195/ie.2024.386 67 from the dialogue. Shortly after reviewing the data and strategizing the responses from the fall retreat, the coordinator of the program follows-up with the cohort of EdD students who responded to the survey via a Zoom meeting and shares with them the data and action responses. This user-centered strategy (improvement science principle 2) creates a feedback loop between the students and faculty communicating how user inputs shape the ongoing improvement efforts. Preliminary results from the two surveys in the area of social justice suggest areas of strengths and areas for growth. For example, one student noted in their survey response that a strength of the program was its “emphasis on social justice and equity and need to bring about change in educational systems,” while another noted that the program encouraged them to “question the status quo to better understand how the root cause of problems may be exacerbating inequity and social injustice.” In terms of areas of growth, one student shared that “[c]lasses on equity need to expand beyond race. We had one class that included immigrants and LGBT, but it was asynchronous and provided no room for us to talk about some of the issues facing our schools (transgender students and locker rooms, Title IX and girls/boys sports teams, students and their pronouns and school's responsibilities for parent notification). Seems we missed out on an opportunity to learn more about a wider lens of equity issues on our doorstep.” Overall, the findings suggest areas to revise current courses’ learning experiences, opportunities for new course development to address the voids and opportunities to more effectively integrate DEI across EdD coursework and programmatic expectations (e.g., dissertation in practice). Understanding the Improvement Area: Empathy Interviews Empathy interviews are a strategy used to ascertain stories and experiences of people who are connected to an improvement area, and they help to learn more about variation in the system and how the system is working (Carnegie Foundation for the Advancement of Teaching, 2018; Hinnant-Crawford, 2020; High Tech High, 2023; Smith, 2020). The improvement team utilized empathy interviews with three EdD program faculty members and one student to thoroughly understand the area of improvement as well as to identify potential areas to pursue to make effective changes. Specifically, empathy interviews were used as an opportunity to provide the EdD faculty improvers successful learning activities related to DEI, faculty needs, and considerations. As empathy interviews go, the conversations were brief, each only 10-15 minutes, but revealed (1) how people felt successful integrating DEI into class instruction, learning activity, or a successful learning moment and (2) suggestions for what faculty members might need, do, or keep in mind for deliberate incorporation of DEI into the EDSI EdD courses. Pedagogical design ideas for consideration included the incorporation of purposeful case studies and learning opportunities designed to invite authentic connections to context/system, practice, policies, or people. These ideas will be shared with EdD course faculty as they engage in continuous improvement efforts for their courses. Further, the empathy interviews also provided insight for what program and department leadership should consider to support their faculty. For instance, making space for faculty to discuss and come to common understanding of expectations and goals related to DEI as it specifically relates to the program is one way to provide faculty with opportunities for improvement. Understanding the Improvement Area: Root Cause Analysis A fishbone diagram is one way to organize a root cause analysis of an improvement area as it captures potential causes leading to a particular effect (Hinnant-Crawford, 2020; Institute for Healthcare Improvement, 2023). The fishbone contains the improvement area (effect) in the head and the elements (causes) that lead to or exacerbate the problem become the different thematic bones of the fish with specific related causes listed under each. For instance, since the improvement focus was integrating DEI more explicitly and effectively in the EdD program, the bones of the fish could include themes connected to this problem of practice such as Program Coherence or Faculty Experience. The fishbone diagram develops iteratively, because as the team learns more about the improvement area, the new learning is incorporated into the fishbone (e.g., see LaRosa, 2021). The fishbone below was developed and updated to reflect new understandings from program-related data. Figure 1. Fishbone Diagram Note. An organizational cause-and-effect schemata such as the fishbone helps the team see specific target areas for targeted improvement efforts to integrate equity and social justice learning opportunities across the EDSI EdD. Tran et al. Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 9 No. 1 (2024) DOI 10.5195/ie.2024.386 68 Phase 3: A Path Forward for EDSI EdD Program Improvement The EdD students have experienced seismic shifts in the P-20 educational contexts, especially post-pandemic. Programs that prepare scholar practitioners to make improvements in those spaces need to be responsive to successful meet their program outcomes and visions for their graduates, making program improvement and redesign efforts imperative. Engaging in program improvement and redesign is a collaborative, iterative, complex endeavor (Young et al., 2022). The future improvement work for the EDSI program change efforts (current and future) will map onto the recommendations provided by Orr and colleagues (2013) and Young and colleagues (2022) related to engaging in program redesign. Scholars who have done work in this area note that the process involves preparing for redesign through understanding of the current program–which we outline above, building a team of key stakeholders who are willing to engage in program improvement (EDSI faculty), committing to dedicating the necessary resources (e.g., time, space, funding, talents) to the effort, and charting a path forward that embraces an inquiry approach and is dedicated to monitoring and celebrating the improvement efforts (big and small) throughout the process (see Orr et al., 2013; Young et al., 2022). Program improvement and redesign includes developing a profile of an exemplary graduate of the program (similar to the future iterations of EDSI’s program outcomes and leadership dispositions). Aligned with the research on backwards design (McTighe & Wiggins, 2012; Reynolds & Dowell Kearns, 2017; Wiggins & McTighe, 1998), clarity on who a graduate is and what they are able to do (i.e., improve educational systems and structures), inspires the next phases of program improvement, which involves determining the organizational complements that need modification, adjustment, or redesign to facilitate powerful, transformative learning for students in order to create the educational environment for students to develop successfully (Young et al., 2021). When it is clear to program faculty who the graduates of the program are and what the program outcomes and goals, and leadership dispositions are, then areas like student recruitment and selection for admission can also become more purposeful. Researchers in program redesign and improvement also recognize that this work is never done. Continuous improvement is not another thing to do, it is the work of preparing leaders itself. Over the course of its three-year existence, the EdD program has aimed to make adjustments and positive change towards equity based on multiple sources of information (e.g., faculty and student feedback and evaluation). More recently, the fishbone development captured above highlights improvement areas that are both changeable and in the direct control of EDSI EdD faculty. For instance, some of the sub-bones under the program structure, faculty learning, course development, and student experience themes are and will continue to be the first areas for targeted improvement. Table 1 below includes some of the key findings synthesized from the findings from syllabi review, empathy interviews, et cetera in the left column and suggestions for improvement that stem from professional recommendations and existing research in the right column. Items that are crossed out indicate past or current efforts by the team. Non-struck through items are for future consideration. Table 1. Program Understanding Results and Suggested Steps for Syllabus and Course Refinement Program Improvement Finding Potential Response Action Steps EdD faculty hold varying understandings of the program outcomes as evidenced by a lack of clarity and focus.  Program faculty collaborate to rewrite program outcomes  Participate in a series of professional learning opportunities to build capacity in program understanding including improvement science and dissertation and practice clarity  Develop a dissertation in practice (DiP) guide  Update admissions decision procedures to reflect program purpose and outcomes  Identify and higher faculty members who can deepen the educational experiences of our scholar practitioner EdD students Lack of clarity around dissertation in practice process, organization, and expectations.  Communicate a focus for equity and social justice DiP focus  Faculty will plan how to support the dissertation process  Develop a dissertation in practice guide Overarching program outcomes are not fully articulated, operationalized, and/or communicated to syllabi creators.  Plan opportunities to come to a collective understanding of program outcomes and leadership dispositions  Develop a set of EdD leadership dispositions for the program that reflects the core values of the program and how graduates should approach leadership  Include leadership dispositions in EDSI EdD syllabi  Make intentional connections between program outcomes and course activities and outcomes  Continue to develop the program’s online learning informational resource hu  EDSI EdD Handbook revisions/update  Dissertation in practice outline development Equity topics in courses were limited.  Equity focus includes topics such as race, gender, (dis)ability, LGBTQIA+, nationality, religion, race, ethnicity, citizenship, language, and beyond  Intentional selection of course materials to support the foundation of systems improvement through justice, equity, and inclusion  Make syllabi revisions to incorporate an intersection of systems improvement and equity  Incorporate opportunities to bridge learning about DEI topics and values and authentic connection to practice, policy, and people (e.g., case development for student individual or collaborative engagement) One course clearly addresses how the class will further the students' understanding of district, state, and national policies. • All courses incorporate policy at different levels (i.e., local, state, federal) Faculty need common understanding of what DEI in the EdD program looks like. • Integrate dedicated, regular time and processes for collaborative discussions to develop this understanding • Emphasize diversity and equity-oriented improvement and outcomes for the DiP by EdD students to lead improvements directed towards disrupting problematic systems to promote equitable and socially just practices Need for regular evaluation of the program outcomes so that it is culturally responsive to current societal challenges. • Integrate dedicated, regular time and processes for purposeful continuous improvement • Intentional improvement of syllabi informed by syllabi evaluation protocols that reveal if the equity curriculum is introductory, advanced, or mastery Improving Equity Focus in an EdD Program Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 9 No. 1 (2024) DOI 10.5195/ie.2024.386 69 Broadly, three program improvement foci are included in the improvement arc: program purpose, curricular design, and continuous improvement processes. Program Purpose EDSI EdD program faculty need to refine the program outcomes and leadership dispositions to describe an EdD graduate more clearly and to emphasize a justice, equity, and inclusion lens. These then can be applied to coursework to develop students’ skills in leading change to dismantle, eliminate, and then improve institutional and individual injustices. To this end, the program faculty recognize they need to be prepared to help students identify and combat inequities and oppressive systems. Thus, the faculty will collaborate to develop outcomes and adopt leadership dispositions to be more relevant and more explicitly address dismantling inequities through continuous improvement work and aligning to the College of Education’s overarching goals of diversity, justice, equity, and inclusion. A clear sense of program purpose and who students will be upon graduation helps calibrate admissions decisions (Young et al., 2022). Aligned with the focus of the revised program outcomes and leadership dispositions, the EDSI EdD faculty are working to further refine their EdD cohort member selection through updating student admissions criteria and application materials. For example, based on the emphasis on social justice and equity, applicants will now include a description of their potential to positively impact social justice and equity outcomes in their personal statement for the next admission cycle (2023-24). Application reviewers will determine whether the description defined by the applicants aligns with the values defined by the EDSI EdD and College of Education to determine this potential. Curricular Design Curriculum cohesion will more effectively support EdD students’ abilities to make bold, equitable system changes in their spheres of influence (captured in their dissertations in practice). With the findings from the process of gaining deeper insight of the program as outlined above, EDSI program faculty can be provided a promising improvement picture of how course syllabi and activities might be revised. Currently, EDSI faculty are actively working to develop new courses and content to align with the program’s equity-focused goals. For instance, in response to needs identified by our preliminary gap analysis, a new course, Inclusive Talent Management: Cultivating Education Workplace Inclusion, was developed to explore how educational leaders nurture diversity, inclusion, belonging, and equity in their P-20 work environments. Students in the course will read topics related to diversity and inclusion (e.g., training, leadership, hiring, micro/macro-aggression, and creating a workplace that promotes belonging) from perspectives that are traditionally marginalized (e.g., gender, race, sexual orientation, ability) in the curriculum, improving equity and social justice opportunities for students and proactively working to diversify hiring and retention. Continuous Improvement Processes It is anticipated that in upcoming semesters, faculty will engage in facilitated, collaborative, and independent opportunities for additional program-related improvements. An important consideration for sustaining continuous improvement is to incorporate professional practice and habits into the regular work and expectations of the faculty (Anderson et al., in press-b). It is necessary to foster the conditions for EdD program faculty to regularly evaluate outcomes so that the program is culturally responsive to local and global societal considerations, including the fallout from the pandemic. Continuous improvement as it relates to program design diversity, equity, and inclusion are a part of EDSI and the Department’s DNA. As this work unfolds and progresses, faculty with the program coordinator will ensure that social justice and equity will be embedded into all of the EdD program’s courses and dissertation in practice processes. From engaging more often in this process, it has become clear that continuous improvement should be woven into the norms of the department's professional practice through existing channels (e.g., monthly department/program meetings). A continuous improvement approach to programmatic design is also being infused throughout the department’s other programs (e.g., MEd principal preparation), and there is thus an opportunity to build off these routines. CONCLUSION In the spirit of continuous improvement, this work is never done. It will be forwarded by dedicating time and resources to continuous improvement efforts, and the user-centered approach will continue to include using a constellation of program-related information for analysis of the current curriculum, gathering additional input from stakeholders connected to the program (e.g., faculty, EdD students), iterating and revisiting the root cause analysis diagram to guide improvement effort focus(es), surfacing opportunities and specific strategies for improving equity and social justice learning experiences for students, and proactively working for inclusive retention of faculty and matriculation of EdD students. As mentioned, the program aims for equity and diversity to be built into the DNA of our program and the EdD, and this emphasis has become increasingly critical in a post-pandemic world where we have seen the exposure and exacerbation of the chasm created by inequity between the privileged and the marginalized. Continuous improvement steps help sharpen program focus and further strengthen a program that cultivates socially just and equitable leaders to make positive impacts in their contexts. REFERENCES Anderson, E., Cunningham, K. M. W., & Eddy-Spicer, D. H. (2023). Leading Continuous Improvement in Schools: Enacting Leadership Standards to Advance Educational Quality and Equity. Routledge. Anderson, E., Cunningham, K. M. W., & Richardson, J. W. (2023). Designing a Multi-Level Learning Trajectory for Sustaining Continuous School Improvement: Lessons for Transformative Organizations. In E. Anderson & S. Hayes (Eds.) Continuous Improvement: A Leadership Process for School Improvement. Information Age Publishing. Bambra, C., Riordan, R., Ford, J., & Matthews, F. (2020). The COVID-19 pandemic and health inequalities. Journal of Epidemiology and Community Health, 74(11), 964–968 Biag, M. & Sherer, D. (2021). Getting better at getting better: Improvement dispositions in education. Teachers College Record, 123(4), 1–42. https://doi.org/10.1177/016146812112300402 Bryan, M. (2021). Diversification Gap Analysis. Office of Diversity, Equity, and Inclusion. University of South Carolina. Bryk, A. S., Gomez, L. M., Grunow, A., & Lemahieu, P. G. (2015). Learning to improve: How America’s schools can get better at getting better. Harvard Education Press. Carnegie Foundation for the Advancement of Teaching (2018). Empathy Interviews. Teaching Commons. https://doi.org/10.1177/016146812112300402 Tran et al. Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 9 No. 1 (2024) DOI 10.5195/ie.2024.386 70 Carnegie Project for the Education Doctorate (2021). Our vision and mission. https://www.cpedinitiative.org/vision-mission Gibson, B., Schneider, J., Talamonti, D., & Forshaw, M. (2021). The impact of inequality on mental health outcomes during the COVID-19 pandemic: A systematic review. Canadian Psychology/Psychologie Canadienne, 62(1), 101. Goudeau, S., Sanrey, C., Stanczak, A., Manstead, A., & Darnon, C. (2021). Why lockdown and distance learning during the COVID-19 pandemic are likely to increase the social class achievement gap. Nature Human Behaviour, 5(10), 1273–1281. Greene, L. S., Inniss, L. B., Crawford, B. J., Baradaran, M., Ben-Asher, N., Capers, I. B., James, O. R., & Lindsay, K. (2019). Talking about Black Lives Matter and# MeToo. Wisconsin. Journal of Law, Gender, & Society, 34 (109). High Tech High Graduate School of Education (2023). Empathy Interviews. https://hthgse.edu/resources/empathy-interviews/ Hinnant-Crawford, B. N. (2020). Improvement Science in Education: A Primer. Meyers Education Press. Institute for Healthcare Improvement (2023). Cause and Effect Diagram. Tools. https://www.ihi.org/resources/Pages/Tools/CauseandEffectDiagram.aspx Kaskazi, A. (2021). The crises of 2020: The effects of intersectionality and virality on marginalized youth in the US. Journal of Children and Media, 15(1), 105–108. Kim, M. J., McKenna, H., Park, C. G., Ketefian, S., Park, S. H., Galvin, K., & Burke, L. (2020). Global assessment instrument for quality of nursing doctoral education with a research focus: Validity and reliability study. Nurse Education Today, 91, 104475. Krouse, H. J. (2020). COVID-19 and the widening gap in health inequity. Otolaryngology–Head and Neck Surgery, 163(1), 65–66. Langley, G. J., Moen, R. D., Nolan, K. M., Nolan, T. W., Norman, C. L., & Provost, L. P. (2009). The Improvement Guide (2nd ed.). Jossey-Bass. LaRosa, D. (2021) Coaching Elementary Educators’ Culturally Sustaining Teaching. [Doctoral dissertation, University of Pittsburgh]. https://d- scholarship.pitt.edu/41486/1/larosaDenise_etd_2021_dschol.pdf McTighe, J., & Wiggins, G. (2012). Understanding by design framework. Alexandria, VA: Association for Supervision and Curriculum Development. Nelsestuen, K., & Smith, J. (2020). Empathy Interviews: An Effective Tool for Understanding the User Experience. Virtual Presentation at the Carnegie Summit. https://www.carnegiefoundation.org/summit/embedding-equity- in-continuous-improvement/data-measurement-and-equity-of-voice/ Reynolds, H. L., & Kearns, K. D. (2017). A planning tool for incorporating backward design, active learning, and authentic assessment in the college classroom. College Teaching, 65(1), 17–27. https://doi.org/10.1080/87567555.2016.1222575 Webb, N. L. (2002). Depth-of-knowledge levels for four content areas. Language Arts, 28(March). http://ossucurr.pbworks.com/w/file/fetch/49691156/Norm%20web%20do k%20by%20subject%20area.pdf Wiggins, G., & McTighe, J. (2011). The Understanding by Design guide to creating high-quality units. ASCD. Young, M. D., O’Doherty, A., & Cunningham, K. M. W. (2022). Redesigning Educational Leadership Preparation for Equity: Strategies for Innovation and Improvement. Routledge. https://www.cpedinitiative.org/vision-mission https://hthgse.edu/resources/empathy-interviews/ https://www.ihi.org/resources/Pages/Tools/CauseandEffectDiagram.aspx https://d-scholarship.pitt.edu/41486/1/larosaDenise_etd_2021_dschol.pdf https://d-scholarship.pitt.edu/41486/1/larosaDenise_etd_2021_dschol.pdf https://www.carnegiefoundation.org/summit/embedding-equity-in-continuous-improvement/data-measurement-and-equity-of-voice/ https://www.carnegiefoundation.org/summit/embedding-equity-in-continuous-improvement/data-measurement-and-equity-of-voice/ https://doi.org/10.1080/87567555.2016.1222575 http://ossucurr.pbworks.com/w/file/fetch/49691156/Norm%20web%20dok%20by%20su http://ossucurr.pbworks.com/w/file/fetch/49691156/Norm%20web%20dok%20by%20su http://ossucurr.pbworks.com/w/file/fetch/49691156/Norm%20web%20dok%20by%20subject%20area.pdf