Microsoft Word - 64 - final layout.docx New articles in this journal are licensed under a Creative Commons Attribution 4.0 United States License. This journal is published by the University Library System of the University of Pittsburgh as part of its D-Scribe Digital Publishing Program and is cosponsored by the University of Pittsburgh Press. 30 This journal is supported by the Carnegie Project on the Education Doctorate: A Knowledge Forum on the EdD (CPED) cpedinitiative.org impactinged.pitt.edu ISSN 2472-5889 (online) Vol. 3, No. 2 (2018) DOI 10.5195/ie.2018.64 Program Innovation and Design in an Era of Accountability: EdD Faculty Work as a Community of Practice Joy C. Phillips Drexel University joy.phillips@drexel.edu Kathy D. Geller Drexel University kathygeller@drexel.edu Kenneth J. Mawritz Drexel University kjm97@drexel.edu ABSTRACT The essay describes how a task team of Drexel EdD faculty worked as a community of practice (Wenger, 1998) to conduct a crosswalk to align program principles, known as the Keystones, with national and university standards including the six Guiding Principles for Program Design contained in the CPED Framework (Carnegie Project on the Education Doctorate, n.d.-a). In accordance with the study institution’s planning tool, the EdD crosswalk also demonstrates alignment with the National Council for the Accreditation of Educator Preparation (CAEP) Advanced Program Standards (Council for the Accreditation of Educator Preparation, n.d.-a), Drexel University Student Learning Priorities (DSLPs), and Drexel School of Education Program Themes. Discussion examining the task team’s work is framed around the four interlinking components of community of practice: meaning, practice, community, and identity. Wenger (1998) describes these components as “deeply interconnected and mutually defining” (p. 5). While uncommon, the process described in this essay represents the intent of assessment scholars (Banta & Palomba, 2014) and contributes to deepening understanding of higher education assessment work. KEYWORDS: higher education; program assessment; Carnegie Project on Education Doctorate; EdD program design and innovation INTRODUCTION The Carnegie Project on the Education Doctorate (CPED) celebrated its 10th anniversary in June 2017. This milestone provides a backdrop for considering how faculty at Drexel University, a CPED member institution, aligned the EdD program with CPED Guiding Principles, national, and university standards. The alignment work was undertaken as part of a larger effort to review the EdD program’s design and level of curricular innovation. The faculty self- study exercise presented in this essay addresses three themes of this special issue: specific uses of the CPED Guiding Principles for developing and/or evaluating programs, determining graduate outcomes, and innovative ways to redesign and improve EdD programs. The manuscript describes how a task team of Drexel EdD faculty worked as a community of practice (Wenger, 1998) to conduct a crosswalk to align program principles, known as the Keystones, with national and university standards including the six Guiding Principles for Program Design contained in the CPED (n.d.- a) Framework. In accordance with the Drexel planning tool, the EdD crosswalk also included alignment with other key standards: The National Council for the Accreditation of Educator Preparation/Advanced Program Standards (Council for the Accreditation of Educator Preparation [CAEP], n.d.-a), Drexel University Student Learning Priorities (DSLPs), and Drexel School of Education Program Themes. The CPED Framework provided a lexicon and pedagogical constructs that align well with the Drexel EdD Program Keystones in which students are guided to become scholarly practitioners who address complex problems of practice through inquiry by producing a dissertation focused in practice. This manuscript contains four major components: study context; the alignment work: description of self- study; working as a community of practice: discussion; and conclusion with next steps. STUDY CONTEXT The self-study is grounded in three interrelated contexts: CPED, the push for higher education assessment and accreditation, and Drexel School of Education’s quest for CAEP accreditation. Collectively, these contexts situate the essay into the larger national Program Innovation and Design in an Era of Accountability: EdD Faculty Work as a Community of Practice Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 3, No. 2 (2018) DOI 10.5195/ie.2018.64 31 discussion of the need to demonstrate that higher education students graduate with measurable evidence of having obtained program learning outcomes. Carnegie Project on the Education Doctorate In CPED’s first decade, a growing number of CPED institution faculty members have answered the call to radically revise their EdD programs, and research papers and resources have begun to emerge (CPED, n.d.-b). Much of the early literature focused on descriptions of how member faculty have engaged in the redesign process and how the redesigned programs function. As the CPED initiative moves into its second decade, more in-depth research is developing (Perry, Zambo, & Wunder, 2015). This paper contributes to the dialogue by exploring how member faculty at Drexel University are engaging in “conversations that matter” (reference to a university assessment conference) as they analyze if and how the CPED framework fits an existing EdD program of study. As described in the next section, the impetus for this work is a response to intense public and private pressure to provide evidence of dynamic student outcomes. Higher Education Assessment and Accreditation The increasingly stringent demands for higher education accountability manifest through assessment and accreditation processes. Gradually, accreditation organization leaders have adopted standards, and higher education administrators have strengthened, deepened, and broadened institutional effectiveness programs, policies, and staff (Field, 2006). In July 2013, two competing educator preparation accrediting organizations, the National Council for Accreditation of Teacher Education (NCATE) and the Teacher Education Accreditation Council (TEAC), merged into a single new professional organization: known as CAEP (n.d.-b). In July 2016, the CAEP Board of Directors adopted revised standards for Advanced Programs (CAEP, n.d.-c). The new Standards for Advanced Programs apply to any higher education institutions seeking CAEP accreditation after September 1, 2017 (CAEP, n.d.-c). Because the Advanced Program standards are new, most higher education institutions have not yet engaged in this level of program accreditation. Indeed, CAEP administrators are continuing to make policy changes (CAEP, n.d.-d). All CPED member institutions seeking CAEP accreditation will need to comply with these new program standards. While the gradual adoption of more rigorous university accountability processes has made assessment standards and procedures more visible, most faculty still interpret institutional assessment as something “done to” them, rather than as a tool for them to use for program improvement (Haviland, 2014, p. 755). Evidence of exemplary postsecondary assessment practices continues to be limited (Haviland, 2014), which makes the account discussed in the essay noteworthy. Drexel University’s Quest for CAEP Accreditation The third contextual dimension includes an overview of the Drexel University School of Education and a description of the faculty self-study task team. This essay details how the Drexel self-study is situated in the CPED and higher education assessment environments. The university. This self-study was conducted by faculty from Drexel University (DU), a private, non-profit institution, whose brick and mortar campus is located in the center of Philadelphia, Pennsylvania. Currently, the School of Education (SoE) has just over a thousand undergraduate and graduate students, with 42 faculty serving in and across 43 programs that confer 17 types of degrees (DU/SoE, n.d.-a). The SoE offers both an EdD and a PhD programs. While the PhD program is full-time and on-campus, EdD students may participate either completely online or in a hybrid (60%/40%), executive weekend format. The EdD part-time program of study is designed to be completed in three years or 12 quarters. Although the Drexel EdD program was launched in 2009, the university did not become a CPED member until 2015. Thus, the EdD program was fully developed when the faculty self-assessment process began. As confirmed in the university’s CPED application and participant responses from an October 2016 Convening presentation, the university’s EdD program design exhibits a robust demonstration of the CPED intent as described in the CPED Framework. The Drexel EdD program was designed for part-time working professionals from a variety of disciplines including K-12, higher education, non-profit leadership, health care, and industry. From the outset, the EdD program sought to develop students as scholar practitioners who would conduct their dissertation inquiry on a problem of practice. Therefore, the CPED definition of education doctorate is directly aligned with the Drexel EdD. The EdD program definition is also aligned with the Drexel Mission Statement, which focuses on productive professional and civic lives and collective expertise on solving society’s greatest problems (DU, n.d.). The program’s applied orientation is consistent with the region’s historical roots in practical education. The goal of the Drexel EdD dissertation is for students to develop an actionable response to a critical educational need; in contrast, the PhD program prepares education leaders who aspire to be researchers, scholars, and instructors in higher education (DU/SoE, n.d.-b). Faculty self-study task team. The self-study began in January 2017 when the SoE administrators advised faculty to begin preparing for accreditation from the CAEP; the actual external review is scheduled to occur in 2020-2021. The CAEP accreditation will accompany the Drexel University’s regional accreditation with the Middle States Commission on Higher Education (MSCHE). MSCHE includes the geographic areas of Delaware, the District of Columbia, Maryland, New Jersey, New York, Pennsylvania, Puerto Rico, and the Virgin Islands (MSCHE, n.d.). To launch the self-study, the EdD Program Director announced the accreditation initiative to the EdD Advisory Committee. The Advisory Committee includes an average of seven faculty members, most of whom are non-tenure-track clinical faculty. At Drexel, most clinical faculty appointments are full-time and include faculty voting rights, service obligations, and university benefits. Responding to a call for volunteers, the Program Director and two faculty colleagues, all clinical faculty, formed a task team to create a preliminary EdD program assessment plan. The faculty task team was dispersed across three states (e.g., California, Pennsylvania, and North Carolina). Two of the three faculty members were physically located away (i.e., remote) from the university main campus. The self-study process was conducted electronically (e.g., web-based video platform Zoom, email, Word documents, and phone calls). Phillips, Geller, & Mawritz Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 3, No. 2 (2018) DOI 10.5195/ie.2018.64 32 Each member of the team brought a specific perspective to the work: The Program Director had a corporate training background leading global management development efforts and years of experience teaching in the Drexel EdD program. The second faculty member had extensive K-12 leadership experience followed by a longstanding faculty appointment in the EdD program. The third member, who was new to the university, had multi-disciplinary leadership experience, including 20 years in higher education and expertise in graduate-level program assessment. The assessment work required the faculty task team to engage in an array of tasks including broadly conceptualizing the coursework programmatically, focusing on minute details of coursework, identifying connections between and among courses, raising the level of analysis to the program level, and comparing/contrasting discrete sets of standards. Moving through discussions within, between, and across the array of intense cognitive tasks required the team to invest a considerable amount of time and energy. The team’s work is described in the next section. THE ALIGNMENT WORK The EdD task team began their work with the SoE EdD Keystones. The Keystones are so named because Pennsylvania’s nickname is the Keystone State. Developed by the faculty who created the EdD program, the Keystones provide a vision of the desired EdD graduate. The Keystones were not designed to easily identify specific measurable student outcomes, which was at odds with assessment requirements. To address this problem, the self- study task team operationalized the five Keystones with more detailed mastery descriptions of student outcomes (see Table 1). Table 1. School of Education EdD Keystones with Outcomes Keystone/Program Learning Outcome Outcomes 1. EdD graduates possess the abilities to create and support communities that are the basis for sustainable change. Graduates demonstrate mastery to: (a) create and nurture networks of people (b)establish collaborative partnerships (c) work effectively in communities of practice (d) honor diversity (e) lead with a global mindset 2. Leaders develop the habits of mind and competencies to lead complex organizations shaped by global forces. Demonstrate mastery for leading systemic change drawing from a range of theoretical frameworks (a) Systems Thinking (b) Adaptive Leadership (c) Theory U (d) Creative Problem Solving (e) Design Thinking 3. Leaders develop the abilities to sustain their own leadership growth. Each student delineates and identifies existing assumptions and mental models that inform both leadership practice and scholarly inquiry; and develops the mastery to suspend judgment and bracket biases (epoché) 4. EdD graduates utilize the full range of emerging technologies to reach across generations, communicate effectively, and engage others in meaningful change. Demonstrate mastery in the: (a) use and integration of learning technologies (b) appropriate use of technology in general to communicate, collaborate, and resolve problems 5. EdD graduates exemplify the curiosity, inquiry skills, and scholarly competencies needed to investigate an idea and transform it into meaningful action. Demonstrate the mastery to identify a “problem of practice” and conduct scholarly research leading to an actionable solution. After developing the Keystone mastery statements, the team struggled with how to begin the assessment process. Following considerable conversation, in which the team wrestled to distinguish course from program learning outcomes, the team decided to begin by going deep into individual program courses. This decision provided an open path for further discussion and analysis because the course learning outcomes were clearly documented in course syllabi and familiar to the faculty. Working in an emergent manner, the team created a process as they progressed. Ultimately, the process included examining existing course-level learning outcomes, developing program-level learning outcomes, aligning program-level outcomes with the CPED Framework and national and institutional standards, and identifying measurable outcomes with a review and follow-up process. Each of these is discussed in the following sections. Program Outcomes by Course Initially, the team divided EdD program courses into two categories: leadership (4 courses) and research (9 courses), excluding courses offered in concentration areas. One team member pulled the catalog course descriptions and identified which Keystones were addressed in each course. Another associated the Keystones with the learning objectives by course. Then, the team created a matrix that listed by course: the associated Keystone, the learning goals/student knowledge base, and the student application or evidence of performance. For example, data included formative course-level work that evidenced student performance in creating effective communities through knowledge-based development and application, or student work that evidenced the ability to honor differences through diversity, a global mindset, or a focus on social justice. Program Innovation and Design in an Era of Accountability: EdD Faculty Work as a Community of Practice Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 3, No. 2 (2018) DOI 10.5195/ie.2018.64 33 EdD Program Assessment Planning Tool The EdD Program Director compiled a matrix of Keystone/Program Learning Outcomes (PLOs) into three columns: Data Evidence—what data or evidence could be used to determine whether or not each PLO had been met; Review of Evidence—how program data/evidence would be reviewed; and, Reporting and Follow-up—what the plan is for reporting the results and findings (e.g., strengths and gaps). This early draft was provided to EdD Advisory Committee members who offered no additional comments. At this stage, deciphering the pieces of information needed to successfully address the alignment was challenging. While the task team was familiar with the Drexel Student Learning Priorities (DSLPs), SoE themes, and CPED Guiding Principles, they realized there was a disconnect incorporating the CAEP vision, mission, and goals. This disconnect existed because the CAEP protocols were new and did not include an administrative component for advanced programs until September 1, 2017. Thus, the task team needed to inform themselves about the new standards, then project a vision for how to fit this component into the alignment. Reviewing and Further Developing Data/Evidence, Review, and Reporting/Follow-up Next, the task team revisited the previous work and considered whether other data/evidence might be identified, how this evidence might be reviewed, and what the follow-up process would be. Because the group could not come to a decision on a single set, they decided that two team members would individually create sets of data/evidence, review, and follow-up as a starting place for further team discussion (see Table 2). Table 2. Two Sets of Possible Data/Evidence, Review, and Follow-up Data/ Evidence (A) Data/ Evidence (B) Review of Evidence (A) Review of Evidence (B) Reporting and follow-up (A) Reporting and follow-up (B) Parsing out Connections to Specific Standards For the next activity, the task team returned to the SoE planning tool-directed alignment of learning outcomes with national and university standards. The process began with the team dividing up national and institutional standards to create a crosswalk, or alignment, with each of the five Drexel EdD Keystones. Table 3 illustrates the Keystone alignment with the CPED guiding principles. The resulting work was transferred to a Master Matrix (see Tables 4 and 5), with each standard in a separate column. Because the Master Matrix contained an overwhelming amount of detailed information, the team focused next on completing the crosswalk by reviewing all standards that had been aligned with specific Keystone/Program Learning Outcomes before addressing the Data/Evidence, Review of Evidence, and Reporting and Follow-up. Table 3. Alignment of EdD Program Keystones with CPED Guiding Principles Program Learning Outcomes Standards CPED Guiding Principles Keystone 1 EdD graduates possess the abilities to create and support communities that are the basis for sustainable change. a. Demonstrate mastery: Create communities I. Nurture networks of people II. Establish collaborative partnerships (iii) work effectively in communities of practice b. Honor difference I. Value diversity II. Lead with a global mindset III. Focus on social justice CPED 1. Equity, Ethics & Social Justice 2. Positive Difference 3. Collaboration/ communication Phillips, Geller, & Mawritz Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 3, No. 2 (2018) DOI 10.5195/ie.2018.64 34 Program Learning Outcomes Standards CPED Guiding Principles Keystone 2: Leaders develop the habits of mind and competencies to lead complex organizations shaped by global forces. Learning Outcome: Demonstrate mastery for leading systemic change drawing from a range of theoretical frameworks: a. Systems Thinking b. Leadership & Adaptive Leadership & Theory U c. Creative Problem Solving and Design Thinking d. Communication CPED 1. Ethics/Social justice 2. Positive Difference 3. Collaboration/communication 5. Theory w/systemic inquiry Keystone 3 Leaders develop the abilities to sustain their own leadership growth. Learning Outcome: Students learn to delineate and identify: a. existing assumptions and mental models that inform both leadership practice and scholarly inquiry b. develop the mastery to suspend judgment and bracket biases (epoché) in conducting inquiry. CPED 1. Ethics/Social justice 2. Positive Difference 3. Collaboration/communication Keystone 4 EdD graduates utilize the full range of emerging technologies to reach across generations, communicate effectively, and engage others in meaningful change. Learning Outcome Demonstrate mastery in the: a. use and integration of learning technologies b. appropriate use of technology in general to communicate, collaborate, and resolve problems CPED 2. Positive Difference 3. Collaboration/communication 5. Theory w/systemic inquiry Keystone 5 EdD graduates exemplify the curiosity, inquiry skills, and scholarly competencies needed to investigate an idea and transform it into meaningful action. Learning Outcome Demonstrate the mastery to identify a “problem of practice” and conduct scholarly research leading to an actionable solution. CPED 2. Positive difference 4. Field based solutions 5. Theory w/systemic inquiry 6. Generation of professional knowledge & practice Program Innovation and Design in an Era of Accountability: EdD Faculty Work as a Community of Practice Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 3, No. 2 (2018) DOI 10.5195/ie.2018.64 35 Table 4. Master Matrix with all Options Table 5. Program Learning Outcomes/Keystones with each Standard Set Program Learning Outcomes/Keystones CAEP DSLP CPED SOE THEMES Reaching Team Agreement on Which Standards Aligned Best with Each Keystone During this work, the team engaged in rich discussion punctuated by critical reflection, questioning, debate, reconsideration, and finally agreement. This deeply collaborative discussion resulted in a narrowing of specific standards to fit each Keystone. The agreed upon standards were then collapsed into one column, as originally intended. Condensing Work into 5-Column Matrix To complete this stage of the assessment work, the final phase of the preliminary process involved arriving at task team consensus on a set of Data/Evidence, Review of Evidence, and Reporting and Follow-up. The team returned the analysis process to the 5-column format (see Table 6). With this phase completed, the initial project goals had been achieved, and the draft was ready to be submitted to the SoE Director of Assessment. An appendix provides detail of one fully explicated Keystone. Table 6. SoE Program Assessment Planning Tool Program Learning Outcomes (PLOs) DSLPs/ Standards/ SoE Themes Data/Evidence Review of Evidence Reporting and Follow-up WHAT are the PLOs? PLOs are learning expectations, expressed in active, measurable terms. WHICH Drexel Student Learning Priorities (DSLPs), Standards (InTASC, CAEP, SPA) and, SoE themes align with each outcome? (See list below) WHAT data or evidence will be used to determine whether or not the outcome is met and/or level of impact on P-12 learning? Provide: - Type of measure - Type of evidence - # of students assessed - Where in the curriculum or program is the outcome assessed? - When and how often will the outcome be assessed? HOW will program data/ evidence be reviewed? Identify: - Evaluation tool (rubric; evaluation summary form; statistical tool) - Performance target, achievement level, or benchmark - Who will review the evidence? - What will be the process? - When and how often will the evidence be reviewed? WHAT is the plan for reporting the results and findings (strengths and gaps)? Identify: - To whom will the quantitative and/or qualitative results be reported? How often? - Who will be responsible for reporting the results and findings? For tracking the follow- up? COLLABORATING AS A COMMUNITY OF PRACTICE In the self-study process, the EdD faculty task team worked instinctively as a community of practice (Wenger, 1998). Wenger (1998) describes this conceptualization as a social learning theory with four main components: meaning, practice, community, and identity (p. 5). As Wenger notes, “these elements are deeply interconnected and mutually defining” (p. 5). Theorized as a matrix with four quadrants: social learning takes place at the “intersection of [these] intellectual traditions” (p. 12). To emphasize the role of community and interaction, each component’s definition begins with CAEP Standards DSLPs/ Standard/ SoE Themes CPED Principles Data/ Evidence (A) Data/Evidence (B) Review of Data/ Evidence (A) Review of Evidence (B) Reporting and Follow-Up (A) Reporting and Follow-Up (B) Phillips, Geller, & Mawritz Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 3, No. 2 (2018) DOI 10.5195/ie.2018.64 36 “a way of talking” (p. 5). The discussion in this section provides an example of Wenger’s model in action, as the task team took an opportunity to: “Articulate a familiar phenomenon [the EdD program] …to push our intuitions: to deepen and expand them, to examine and rethink them” (p. 7). Meaning First, Wenger (1998) describes the component of meaning as “a way of talking about our (changing) ability—individually and collectively—to experience our life and the world as meaningful” (p. 5). As the task team began conversation, they were initially equipped only with the SoE Program Assessment Planning Tool; The first of many challenges associated with the project was making sense of the Tool and the task ahead. All three team members began the work with a sense of commitment to the EdD program and an expectation that the task could be completed in a couple of meetings. Convening the group, the Program Director (PD), a former human resource development leader, was looking for people who had the needed talent and skills. Although the PD had extensive knowledge of the program and its courses, she had little experience with assessment. The faculty member with K-12 experience was deeply familiar with the creation and development of the EdD program from its inception in 2009; from this position, he was well prepared to provide details about the program’s leadership courses and the founders’ intent. As a newcomer, the third faculty member perceived her role as one of asking questions about the program, pausing the conversation for reflection, and interjecting assessment ideology. While initially this approach was meant to further her own understanding, the team agreed that the reflection process also contributed to promoting group understanding and deepening the conversation. The original assignment from the SoE Director of Assessment was to complete what appeared to be a very simple form. However, the task team realized at the first meeting that the challenge was complex. In addition to figuring out how to use the form, the team realized they would need to build out the content by connecting the boilerplate information to the EdD Program mission and the program’s five Keystones. Eventually, the task team reviewed the in- progress work with the Director of Assessment to ensure that it met expectations. In fact, the Director indicated that the work exceeded expectations. Throughout the work, the team pressed themselves to continue the sensemaking process. Within and between each round, the team was faced with the same challenge; how to convert the tasks into meaningful chunks of work that would contribute to their understanding—individually and collectively—and would provide the information necessary to form a foundation and structure for program assessment that could eventually be shared with EdD Advisory Committee and the other faculty teaching in the EdD program. This task never became easier. The more time and energy the team devoted to the project, the more important it became to the team to view the work as something worthwhile that would improve/enhance the EdD program. Equally as important, was keeping in mind that eventually this work would be shared with the larger group who would need to engage in their own sensemaking and might very well disagree procedurally and philosophically. The task team identified the problem to be addressed as how to build upon the foundation Keystones of the EdD program to develop a comprehensive plan of EdD Program Learning Outcomes. This approach required the team to consider the EdD program as a system. Viewing the program as a system was consistent with the program’s use of Theory U (Scharmer, 2016), which is used as a guiding framework for all four leadership courses in the EdD program. Viewing the program as a system required the team to shift back and forth between the fundamental beliefs of the faculty who developed the EdD program, the team’s mission to operationalize the somewhat abstract program Keystones into measurable learning outcomes, and the University’s mission to develop the student’s knowledge base and provide creative application activities. While the team found that the Keystones were generally well- aligned with the CPED Guiding Principles, they also discovered that one principle was not clearly visible: social justice. The CPED Framework indicates that “[the Professional doctorate in education] is framed around questions of equity, ethnics, and social justice to bring about solutions to complex problems of practice” (CPED, n.d.- a, para. 5). Recognizing this alignment gap, the task team reflected often and at length about whether there was a distinct social justice orientation to the program’s EdD students’ dissertation topics. This discussion led to a decision to conduct a topical analysis of the EdD student dissertations. While discussion of the results of that analysis are beyond the scope of this essay, the development illustrates that faculty recognition of the need for further data collection and analysis grew with each conversation. Although this university pulls heavily from students in the mid- Atlantic coast region, it also attracts students from across the United States, including immigrants to this country, and has a growing group of international educators who are attending. The EdD student body is well represented across ethnic groups, and many EdD students of color—as well as others—focus on problems of practice related to educational issues of underrepresented minorities. Other EdD students focus on issues related to students in poverty and first- generation college students. The task team determined that the CPED social justice principle is represented in the EdD program and that they needed to make such language explicit in the developing plan. Discussion also ensued about whether a sixth Keystone should be added that explicitly names social justice; the team decided to table that discussion to future convening with the full EdD Advisory Committee and with other faculty who teach in the EdD program. Practice Second, Wenger (1998) describes the component of practice as, “a way of talking about the shared historical and social resources, frameworks, and perspectives that can sustain mutual engagement in action” (p. 5). Although the task team members were colleagues, they had never worked together on such a complex project. As will be discussed in this section, the group developed an uncommon set of faculty practices for this work. In an approach fostered by the PD task team leader, all three faculty had an equal voice in the conversations, decisions, and work. Except for one occasion, the full team convened for every discussion. This situation, in which two members talked about future steps, led to a critical incident that could have destroyed the group. Although the situation was heatedly discussed, there was a positive outcome: an explicit conversation about the group’s norms. Naming and discussing the issue actually made the group stronger. Program Innovation and Design in an Era of Accountability: EdD Faculty Work as a Community of Practice Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 3, No. 2 (2018) DOI 10.5195/ie.2018.64 37 Chief among the now explicit norms was the shared understanding that the group would work collaboratively, honor each other’s contributions respectfully, and allow unlimited “push-back” from any member on any point at any time. This practice caused the development process to extend over six-months and nearly a dozen one and a half- to three-hour meetings. Such a time commitment strained the patience of all team members who struggled to accommodate the project into a workload already replete with faculty responsibilities. The process was messy and complex, and demanded that team members engage in sometimes head-banging critical thinking. At any time, a team member could and did return to earlier discussions and versions of the developing plan as new, contradictory, or explanatory thoughts occurred between meetings. “But what about?” queries were commonly raised. These new contributions caused the team to work often in a one-step forward, two-step backwards manner. As team members’ understanding grew with construction of meaning of what the process was intended to yield, prior efforts were perceived differently; some previously rejected ideas were resurrected into fresh insights of what elements the plan should hold. At times, the conversations turned philosophical, as the team pondered whether and how the EdD program’s guiding theory, the Keystones, could be operationalized. These philosophical debates prompted one person to ask if “we are making this [process] too complicated,” and another to suggest, “We do not want to debate how many angels can dance on the head of a pin, do we?” in reference to a medieval-era argument about focusing on irrelevant details. But, the question of how to decide what was irrelevant was baffling. The team’s process was the opposite of linear, unfolding in a spiral and iterative fashion despite one member’s mounting frustration with delayed closure. Only in retrospective analysis could the task team’s work be perceived as having occurred in distinct phases. Community Third, Wenger (1998) describes community as “a way of talking about the social configurations in which our enterprises are defined as worth pursing and our participation is recognizable as competence” (p. 5). As described in this essay, the larger community in which the project is situated can be conceptualized as three concentric circles. At the center is the work conducted by the original task team; the next circle is the membership of the EdD Advisory Committee; and the outer circle is the remaining group of faculty who teach in the EdD program. Many of those in the latter group teach primarily in the PhD program. While this essay focuses on the original task team, the team members were constantly aware that at some point the conversation would need to go forward to the larger groups. This recognition was infused in the task teams’ conversations and strengthened their resolve to produce an outcome that could both stand up to scrutiny and be permeable enough to continue to grow and develop. The team’s process of working as a community occurred as follows. At each meeting, one person would come in with an idea from reflection since the previous meeting. Since each person came with different perspective, massive discussion was needed to arrive at a common understanding. The result might be similar or different to the presenting idea, but it would almost certainly have incorporated some aspects of the original idea. The team members at any point could have abandoned the process of meaning making and simply put words on the form, but that did not happen. The team members shared the belief that the task was about improving the EdD program. From filling out a form, the process of complex meaning making emerged. The deeper into it the team went, the more they became convinced of its value to the program. This conviction allowed the team to deconstruct the program, identify data and evidence about the program, and consider how to improve the program. In using this approach, the team practiced another idea promoted in the EdD leadership courses: Leave the dance floor and look at the program from a balcony perspective (Heifetz, Grashow, & Linsky, 2009). In other words, the team needed to step back from their individual faculty work in the program and look at the assessment process from a system’s perspective. To further the goal of developing a meaningful product, the team shared ownership of ideas, and the resulting work exhibited collective meaning. For example, a presentation of the work was accepted for the fall 2017 CPED Convening; however, one member had a scheduling conflict. The group figured out a way to include the third member by arranging with the conference staff to use Zoom. By way of this online video platform, all three task team members conducted the presentation, responded to robust questions and engaged in discussion with the session participants. Towards the end of this major stage of work, the team reflected on their collaborative work. All agreed the work probably could have been completed more quickly without the extensive, intense conversations that mattered. However, the team acknowledged that the alternate result would likely have yielded less common agreement and less complexity of thought and may have faced greater resistance when introduced to the larger EdD faculty. Identity Finally, Wenger (1998) describes identity as “a way of talking about how learning changes who we are and creates personal histories of becoming in the context of our communities” (p. 5). By working collaboratively in this community of practice, the team members strengthened their assessment knowledge and skills individually and as a group. From an uncertain starting point, the members confidence in their work grew; early conference presentations led to more presentations and an invitation to author a piece for a national assessment website. From novices, the team began to see themselves as having important information to share with other university faculty. The task team practiced and reviewed their team interaction responsibilities from the start. These responsibilities required that the team’s practice also take consideration of and respect for faculty, staff, and doctoral student life space. Lewin’s (1936) term “life space” denotes the totality of all the influences on a person at a given moment in time, both the outer environment and inner personal environment (pp. 73-77). This idea is another concept discussed in the EdD leadership courses. Thus, recognizing the micro team effort of the task group, as well as the macro team effort regarding the entire faculty, staff, and study body, prevailed throughout the entire planning process. Finally, in review of the process for program assessment, it is significant to note that key leadership principles related to inclusion, carefully listening to others, and general collaboration were incorporated. As Scharmer (2016) argues, “While management is Phillips, Geller, & Mawritz Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 3, No. 2 (2018) DOI 10.5195/ie.2018.64 38 about ‘getting things done,’ leadership is about creating and cultivating the larger context—the fertile common ground and soil—in which things can happen.” (p. 72). The task team was problem- and EdD-student-focused and guided by a systems approach to thinking and action. The task team also endeavored to identify and expound on both formative and summative data/evidence from the existing program for the developing plan. Much discussion focused on distinguishing formative data (i.e., mainly course-level) from summative data (i.e., program-level) and whether to propose that data systems of credible evidence be created at the formative and/or the summative level. A recurring discussion was to what extent the developing assessment plan needed to be a “polished” document before seeking wider faculty input and discussion. Ultimately, the team agreed that the document language could include planned action steps in addition to incorporating existing data collection and review systems. This decision left the process open to experimentation and discussion with other faculty with the idea that plan refinement (CPED, n.d.-a) would occur in continuous improvement cycles. CONCLUSION In sum, this manuscript demonstrates how faculty in a CPED- affiliated EdD program responded to external accountability demands by embracing the goal of developing, implementing, and measuring program learning outcomes consistent with CPED Guiding Principles, national CAEP Standards, Drexel student learning priorities (DSLPs), and SoE themes. Although spurred by external accountability demands, EdD program faculty took this opportunity to create the baseline for an EdD program assessment plan that can serve as the framework for cycles of continuous program improvement. Trying but failing to create a linear working process, the faculty task team kept the EdD program’s Keystone principles at the center of the emergent assessment work. The team used the Keystones throughout their work to create a preliminary EdD program learning outcomes document. The document included identifying specific, measurable learning outcomes as data/evidence and accessing or creating systems for regular review of evidence and follow-up actions. To complete this project, the team worked as a community of practice. As is evident from the discussion, the four components of Wenger’s (1998) community of practice are threaded within and across the descriptions of each element and its application to the team’s work, thus, demonstrating that “these elements are deeply interconnected and mutually defining” (p. 5). The next step in this process is for the task team to take the preliminary work to the EdD Advisory Committee and subsequently to the full EdD faculty for discussion. Successful transfer of this process to the wider group is incumbent upon task team members engaging in and fostering a culture and norms aimed toward continuous program improvement. Throughout the ongoing, now normalized, continuous program improvement work, the EdD task team faculty vowed to keep the CPED Guiding Principles tied tightly to program development and evaluation, to continue to identify and examine graduate outcomes aligned with CPED principles, and to maintain the EdD program’s focus on innovative design by ensuring that the program will stand up to both internal and external assessment scrutiny. REFERENCES Carnegie Project on the Education Doctorate (CPED). (n.d.-a). About us. Retrieved from http://cpedinitiative.site-ym.com/page/AboutUs Carnegie Project on the Education Doctorate (CPED). (n.d.-b). Research and resources. Retrieved from http://cpedinitiative.site- ym.com/page/researchresources Council for the Accreditation of Educator Preparation (CAEP). (n.d.-a). About the CAEP standards for advanced-level programs. Retrieved from http://caepnet.org/standards/standards-advanced-programs Council for the Accreditation of Educator Preparation (CAEP). (n.d.-b). History of CAEP. Retrieved from http://caepnet.org/about/history Council for the Accreditation of Educator Preparation (CAEP). (n.d.-c). Standards for advanced-level programs. Retrieved from http://caepnet.org/standards/standards-advanced-programs Council for the Accreditation of Educator Preparation (CAEP). (n.d.-d). Policy changes: Accreditation for advanced programs. Retrieved from http://caepnet.org/~/media/Files/caep/accreditation/policy-changes-for- advprepprograms.pdf?la=en Drexel University (DU) (n.d.) About Drexel: At a glance. Retrieved from http://drexel.edu/about/glance/mission/ Drexel University/School of Education (DU/SoE) (n.d.-a) Welcome to Drexel University School of Education: Drexel is different. Retrieved from http://drexel.edu/soe/ Drexel University/School of Education (DU/SoE) (n.d.-b) Academics: Doctoral degree programs. Retrieved from http://drexel.edu/soe/academics/doctoral/ Field, K. (2006, September 1). Uncertainty greets report on colleges by U.S. panel. The Chronicle of Higher Education, A1. Haviland, D. (2014). Beyond compliance: Using organizational theory to unleash the potential of assessment. Community College Journal of Research and Practice, 38, 755-765. Heifetz, R. A., Linsky, M., & Grashow, A. (2009). The practice of adaptive leadership: Tools and tactics for changing your organization and the world. Cambridge, MA: Harvard University Press. Lewin, K. (1936). Principles of topological psychology. New York: McGraw-Hill. Middle States Commission on Higher Education (MSCHE). (n.d.) The MSCHE defined. Retrieved from https://www.msche.org/?Nav1=ABOUT&Nav2=MISSION Perry, J. A., Zambo, D., & Wunder, S. (2015). Understanding how schools of education have redesigned the doctorate of education. Journal of School Public Relations, 36(Winter), 58-85. Scharmer, C. O. (2016). Theory u: Leading from the future as it emerges (2nd ed.). Oakland, CA: Berrett-Koehler Publishers. Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. NY: Cambridge University Press. Program Innovation and Design in an Era of Accountability: EdD Faculty Work as a Community of Practice Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 3, No. 2 (2018) DOI 10.5195/ie.2018.64 39 Appendix Articulated Program Learning Outcome (PLO) for EdD Keystone 2 Program Learning Outcomes Standards: DSLPs/CPED/SOE Themes Data/Evidence Review of Evidence Reporting and follow-up Keystone 2: Leaders develop the habits of mind and competencies to lead complex organizations shaped by global forces. Learning Outcome: Demonstrate mastery for leading systemic change drawing from a range of theoretical frameworks: a. Systems Thinking b. Leadership & Adaptive Leadership & Theory U c. Creative Problem Solving and Design Thinking d. Communication CAEP 1. Content & Pedagogical Knowledge 2. Clinical Partnerships & Practice 4. Program Impact 5. Provider Quality Assurance & Continuous Improvement DSLP 1. Communication 2. Creative & Critical Thinking 4. Information Literacy 6. Technology Use 7. Global Competence 8. Leadership CPED 1. Ethics/Social justice 2. Positive Difference 3. Collaboration/communication 5. Theory w/systemic inquiry SOE Themes SOE 1-Innovation & Creativity SOE 2-Diversity SOE 5-Leadership EdD students participation/performance as evidenced by the following: a. Utilize systems thinking to address leadership standards through knowledge base development and application (EDUC 800, 845, 801, 802) • Leadership: individual and team projects • Theory U projects b. Identify & develop leadership style (EDUC 800, 845, 801) • Academic papers & team projects • Technology tools c. Creative Problem Solving & Design Thinking (EDUC 800, 804, 845, 801, 802, 803, 810, 818) • Utilize leadership and research to design dissertation d. Effective communication principles (EDUC 800, 845, 801, 802, 802) • Academic papers & team projects • Discussion boards & voice threads e. Summative evidence includes the following: i. Comprehensive ii. CITI iii. Dissertation Proposal iv. Final Dissertation Create an e-portfolio of student academic and professional accomplishments (e.g., writing assignments, sample discussion boards, team projects, course activities, IRB approval letter, SOE/GS funding letters, juried conference presentations, community presentations, juried publications, etc. ) Comprehensive examination rubric – Leadership; Action oriented research and evaluation; Keystones, Writing and APA - as scored independently by two professors with a rating of 2.0 (scale 0 – 3) or higher on all categories CITI Certification: Create a central repository for CITI Certifications (obtained during EDUC 810) Dissertation Proposal Defense Committee Approval (Forms D-2 & D-2a) Dissertation Final Defense (Forms D – 3 to D- 5) Dissertation Approval Form Completion Form Proposed - Create a review process for the e-portfolio. Semi-annual report (Fall/Spring) Comprehensive Examination outcomes by Program Director Program Manager provides annual listing of CITI certifications for active students Work with HRPP to create IRB approvals annual report for EdD research Create (from D2) Annual report by quarter on doctoral proposal Create (from D5s) Annual report by quarter on doctoral defenses approved