Impacting Education: Journal on Transforming Professional Practice 1 Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 4 No. 1 (2019) DOI 10.5195/ie.2019.85 1 Problems of Practice: A Document Analysis of Education Doctorate Dissertations John C. Gillham, EdD The University of Findlay gillham@findlay.edu Nicole V. Williams, PhD The University of Findlay williamsn1@findlay.edu Gwynne Rife, PhD The University of Findlay rife@findlay.edu Kara K. Parker, EdD The University of Findlay kara.parker@findlay.edu ABSTRACT The purpose of this study was to learn how education doctorate students create the problems of practice researched in their dissertations, and the potential impact of their research on their local contexts to enhance the generation of knowledge. Three research questions guided this study: 1) How do education doctorate students derive their problems of practice?, 2) What is the nature of the problems of practice that the students have studied?, and 3) What are the reported impacts the study of problems of practice has on doctoral students’ local contexts? To answer these questions, the researchers conducted a document analysis of 19 dissertations. Student dissertations included a diverse set of problems of practice largely determined by their professional roles. The findings indicate a need for further refinement of the concept of a problem of practice and how the education doctorate program and their candidates employ the concept of a problem of practice in their dissertations and how this impacts local contexts. Furthermore, the nature of their problems of practice researched through the dissertation contributed to the perceived impact on the local context of the author. KEYWORDS: education doctorate, dissertations, problems of practice According to the National Science Foundation’s National Center for Science and Engineering Statistics (2016), 5,153 doctoral degrees in education (9.4% of the total number of doctoral degrees) were conferred in the United States (U.S.). Distinct from the Doctor of Philosophy (PhD), Schulman, Golde, Bueschel, and Garabedian (2006) noted that the Doctor of Education (EdD), “intended as preparation for managerial and administrative leadership in education, focuses on preparing practitioners—from principals to curriculum specialists, to teacher–educators, to evaluators—who can use existing knowledge to solve educational problems” (p. 26). The Carnegie Project on the Education Doctorate (CPED) further posits that “the professional doctorate in education prepares educators for the application of appropriate and specific practices, the generation of new knowledge and for the stewardship of the profession” (Definition of the Education Doctorate section, n.d.a, para. 1). In an era of increasingly complex contexts and issues facing educational leaders, the application, generation, and stewardship of the scholar- practitioner knowledge and skills attained through the professional doctorate in education is even more necessary. However, in practice there continues to be “a blurring of boundaries” (Schulman et al., 2006, p. 26) between the two degrees, PhD and EdD, and very little difference between the two programs. From its inception, there has been little to distinguish the EdD dissertation from the PhD dissertation (Cremin, 1978, in Perry, 2012). In the early 1990’s, 98% of all College of Education Deans with EdD and PhD programs, reported that dissertations were the culminating experience of their terminal degrees (Osguthorpe & Wong, 1993). In the years since, an emerging body of research has begun to question the appropriateness of a traditional dissertation for practitioner doctoral programs (e.g., Andrews & Grogan, 2005; Deering, 1998; Grogan & Andrews, 2002; Levine, 2005; Malen & Prestine, 2005; Murphy & Vriesenga, 2005; Osguthorpe & Wong, 1993; Shulman et al., 2006; Toma, 2002; Townsend, 2002). In opposition to a uniform dissertation for PhD and EdD students, Hoffman and Perry (2016) argued in favor of the distinction between traditional dissertations for practitioner doctoral programs in that “conceptualizing and addressing problems of practice requires a much different skillset than in a traditional dissertation” (p. 22). For example, Hochbein and Perry (2013) posited that scholar- practitioners utilize three habits of inquiry (decipher, debate, and Gillham, Williams, Rife, and Parker Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 4 No. 1 (2019) DOI 10.5195/ie.2019.85 2 design) to solve their own problems of practice. A central focus of these habits is to utilize the body of research in the field and apply it to a real-world context. Schulman et al. (2006) proposed a Professional Practice Doctorate, a doctoral degree for the practitioner without a dissertation “but with substantive professional assessments at the end” (p. 29). They suggested the culminating experience might include a laboratory of practice, a demonstration of the ability to critically analyze research and an infusion of skills to conduct research and assessment to guide practice. Similarly, Archbald (2008) called for a practitioner doctoral thesis with a distinctive form that should be a part of the institution’s goal to socialize and train effective school leaders. He suggested that an acceptable alternative to the traditional dissertation could be a “distinctive practitioner doctoral thesis” that must include four qualities: 1) developmental efficacy, 2) community benefit, 3) stewardship of doctoral values, and 4) distinctiveness of design (p. 707). Much like Schulman et al. (2006), Archbald (2008) described the need for a problem-based thesis in which “the doctoral candidate identifies and defines a significant problem within his or her organization, collects decision-oriented information, analyzes the problem thoroughly, and develops a feasible plan to solve the problem (p. 714). CPED, a consortium of over 100 universities working together to redesign the EdD degree, proposed a Dissertation in Practice as the new model for the culminating project for the degree. The Dissertation in Practice is “a scholarly endeavor that impacts a complex problem of practice” (CPED, n.d.a, para. 6). CPED does not insist upon a single, specific format but provides members with several models and guidelines for the Dissertations in Practice, including action research, policy analysis, evaluation, modified manuscript model, and group dissertations. As more institutions join the consortium, it is not unreasonable to assume that more forms of the Dissertation in Practice will emerge. STATEMENT OF THE PROBLEM As education doctorate programs continue to explore alternative dissertations to employ problems of practice, the need for further research has emerged to determine how students in these programs define, understand the nature of, and identify the impact on local contexts of their problem of practice research. The purpose of the study was to learn how students in a Doctor of Education (EdD) program, who were predominately employed as educators in the P-12 schools, identified the problems of practice researched in their dissertations, and the potential impact of their research on their local contexts to enhance the generation of knowledge. The following research questions were used to guide this study: 1. How do education doctorate students derive their problems of practice? 2. What is the nature of the problems of practice that the students have studied? 3. What are the reported impacts the study of problems of practice has on doctoral students’ local contexts? REVIEW OF THE LITERATURE Since the inception of the doctorate in education in 1920, experts have sought to exam and differentiate the PhD and the EdD (e.g., Anderson, 1983; Gilbert, Ballati, Turner, & Whitehouse, 2004; Kot & Hendel, 2012; Kumar & Dawson, 2012; Levine, 2005; Mowbray & Halse, 2010; Osguthorpe & Wong, 1993; Shulman et al., 2006; Tennant, 2004; Wergin, 2011). It has been argued that PhD programs prepare individuals to generate knowledge and advance theory (Bowen & Rudenstine, 1992; Golde, 2007; Golde & Walker, 2006). Although discrepancies in the literature exist, most contemporary researchers agree that the EdD emphasizes the importance of the practitioner in the scientist-practitioner model of research, with two key factors distinguishing it from the PhD: the audience and the learning objectives (Wergin, 2011). In contrast, scholars have suggested that EdD programs are predominantly designed to address practitioner challenges that apply to community practice and, in turn, guide system change with evidence, arguments, and values (e.g., Archbald, 2008; Kumar & Dawson, 2012; Maxwell, 2009; Willis, Inman, & Valenti, 2010). Disputing both claims, others have suggested that there are no authentic differences between programs, and have called for the elimination of one, or both degrees (e.g., Clifford & Guthrie, 1988; Deering, 1998; Levine, 2005). To evaluate the differences between the PhD and EdD, Nelson and Coorough (1994) grouped and analyzed 1,000 PhD and 1,000 EdD dissertations by educational content area, and determined that PhD dissertations were more research oriented, whereas EdD dissertations were more oriented to the educational practitioner. Similarly, a study by Walker and Haley-Mize (2012) compared research design, statistics, target population, and significance of results of PhD and EdD dissertations in the content area of special education. No differences were found in the type of degree by gender or applicability of results. However, significant differences were identified on the variables of research design, statistics, target populations, significance of results, age of participants, and exceptionality category. Whereas PhD dissertations were found to focus on theories of research, EdD dissertations were found to focus on theories of practice. Problems of Practice Dissertations The application of practitioner knowledge in a higher- educational context was once referred to as theories of action (Argyris, 1982). Today, however, the concept guiding education doctorate dissertation practices is known as problem of practice (City, Elmore, Fiarman, & Teitel, 2009). Problem of practice dissertations have been described by educational experts as applying questions of pragmatic importance intended to improve educational practice and provide applicable contexts (e.g., Archbald, 2008; Belzer & Ryan, 2013; Maxwell, 2009; Nelson & Coorough, 2006; Willis et al., 2010; Zambo, 2011). Literature has emphasized that problem of practice research combines data and dialogue to create practical solutions (e.g., City et al., 2009). In order to integrate research with practice, Furman (2012) recommended that EdD programs begin to close the gap by training graduate students in the principles of action research and guide dissertation topics to action. Such an approach is supported by the findings of Onwuegbuzie and Dickinson (2006), who identified 27 positive outcomes associated with this method. They found that dissertations providing analytical and conceptual awareness of research applications may lead to more meaningful educational change. Consequentially, practitioners who use action research guide the practice of the future. The functionality and practicality of problem of practice dissertations were also highlighted by Dagenais, Problems of Practice: A Document Analysis of Education Doctorate Dissertations Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 4 No. 1 (2019) DOI 10.5195/ie.2019.85 3 Lysenko, Abrami, Bernard, Ramde, & Janosz (2012). In a thorough review of the literature examining the use of research by practitioners, Dagenais et al. found that the use of research-based information is rarely significant in the current educational practice. However, this problem provides an opportunity for EdD programs in preparing their students. Problem of practice dissertations may allow students to contextually imbed the knowledge transfer process from researcher to practitioner. To address the evolution of problem of practice dissertations, Belzer and Ryan (2013) reported that EdD programs began by focusing on simply improving learning opportunities through research and have evolved into using inquiry, evaluation, and implementation to understand and change current educational conditions. Consequently, when EdD students competently pursue topics intended to promote change from challenge, their work reaps community benefits that have potential to solve local and specific problems, rather than simply spawn generalized findings and focus on future research applications (Archbald, 2008). Guiding the shift towards practical application of research for practitioner benefit is CPED. Carnegie Project on the Education Doctorate (CPED) CPED is a consortium of over 100 schools and colleges of education that seeks to transform and restructure the EdD. It has become the first action-oriented effort in the U. S. aimed at changing the meaning and design of the education doctorate (Hochbein & Perry, 2013). According to CPED (2017) it was essential to develop a framework for EdD programs that “supports creating quality, rigorous practitioner preparation while honoring the local context of each member institution” (p. 1). The CPED framework consists of a definition of the EdD, a set of guiding principles for program development, and a set of design-concepts. In an effort to determine transferability and application at a university level, several studies have examined the CPED framework, problems of practice, and the faculty and students directly impacted by CPED. In order to test the functionality and application of these principles at the university level, Zambo, Zambo, Buss, Perry, and Williams (2014) collected opinions from 296 participants at 14 CPED institutions using an online survey. The study investigated why students and graduates chose to pursue an EdD, what and how they learned, how they viewed themselves, and whether they perceived their programs to be aligned with the CPED principles. Most participants agreed that their university programs were aligned with CPED principles, and pursuing of their degree helped them meet professional goals and develop as scholarly practitioners. Similarly, a study by Ma, Dana, Adams, and Kennedy (2017) analyzed 28 dissertations produced by EdD graduates in order to understand how students derive problems of practice and what impact their studies had on local contexts. The researchers discovered that problems of practice were derived from doctoral students’ felt difficulties and real-world dilemmas. The dissertations evaluated were found to primarily focus on supporting marginalized students or groups, increasing the quality of educator professional development, and supporting novices’ entry into the profession. Furthermore, the study provided guidelines to assist EdD students in deriving problems of practice aligned with the six CPED guiding principles. Rather than adopting a set of exhaustive standards, CPED has developed six guiding principles as a framework for its members’ programs. According to CPED (The Framework, n.d.b. para. 4) the doctorate of education: 1. Is framed around questions of equity, ethics, and social justice to bring about solutions to complex problems of practice. 2. Prepares leaders who can construct and apply knowledge to make a positive difference in the lives of individuals, families, organizations, and communities. 3. Provides opportunities for candidates to develop and demonstrate collaboration and communication skills to work with diverse communities and to build partnerships. 4. Provides field-based opportunities to analyze problems of practice and use multiple frames to develop meaningful solutions. 5. Is grounded in and develops a professional knowledge base that integrates both practical and research knowledge, that links theory with systemic and systematic inquiry. 6. Emphasizes the generation, transformation, and use of professional knowledge and practice. Although dissertations aligned with the CPED principles promote real-world change and educational improvement, ensuring that EdD programs effectively implement CPED guiding principles and design concepts is important. In response to consortium members’ concerns, members of the 2007 CPED Dissertation in Practice Awards Committee conducted an action research study to examine the format and design of dissertations in practice submitted at 25 redesigned colleges/schools of education (Storey et al., 2015). Through analysis of online survey results, interviews, and analyses of dissertations in practice; it was determined that few changes occurred in dissertations after the redesign, despite evidence of change in the dissertation in practice process. Researchers concluded that dissertation submissions lacked clear evidence of impact on practice, and implications for generative solutions at the local and broad context were unclear. These results indicated a further need for rigor and consistency across EdD programs implementing problem of practice research. METHODS As a new member of CPED, researchers who work as faculty in a doctor of education program sought to understand how their students derived problems of practice and what impact their studies had on local contexts (Ma et al., 2017). Three research questions guided this study: 1) How do education doctorate students derive their problems of practice?, 2) What is the nature of the problems of practice that the students have studied?, and 3) What are the reported impacts the study of problems of practice have on doctoral students’ local contexts? Research Design To answer these questions, the researchers conducted a qualitative document analysis of 19 dissertations from the Doctor of Education program graduates. According to Bowen (2009), document analysis is the “systematic procedure for reviewing or evaluating documents - both printed and electronic material” (p. 27). For the purpose of this study, it was necessary to solely rely on the dissertations as the data source to answer the research questions. Gillham, Williams, Rife, and Parker Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 4 No. 1 (2019) DOI 10.5195/ie.2019.85 4 Document analysis provided the researchers with the opportunity to replicate the study of Ma et al. (2017), in which they analyzed 28 dissertations produced by Doctorate of Education students to learn how they derived their problems of practice, and what impact their studies had on local contexts. According to Bowen (2009), the benefits of document analysis also include high efficiency, availability, cost effectiveness, stability, exactness, coverage, and lack of obtrusiveness and reactivity. Setting This study was conducted in a private mid-Western university with a total enrollment of approximately 3,800 students in 2017-18. The mission of the university is to prepare students for to have successful careers as well as meaningful personal lives. The mission of the Doctor of Education program includes preparing educational leaders who are strategic, critically examine policy, practice, and decision making, and develop expertise in a particular subject through scholarly research. The program requires students to conduct independent research that reflects a practical application of a relevant issue and contributes to the profession. During the program, students solve diverse problems using strategic thinking and organizational skills, apply content knowledge and skills in an ethical manner to influence and enhance performance within organizations, and embrace diversity as essential to organizational success, team building, and community partnerships. The doctor of education program in this study admits an average of 20 students per year, who are predominately P-12 public school employees: superintendents, administrators, and teachers. The Doctor of Education program of focus in this study is offered entirely online, with the exception of a three-day face-to-face annual summer institute. Graduates students in the program are required to complete 60 credit hours of core, research, and elective courses with an option to choose elective courses from two concentrations: Teaching and Learning and Superintendent Licensure. They are also required to write a traditional five-chapter dissertation. One of the unique features of the Doctorate of Education program is that the creation of the dissertation is embedded in the research courses. The researchers in this study included the Chair of Advanced Professional Programs in the College of Education, the Chair of the Doctorate of Education program, a faculty member in the Doctorate of Education program, and a student in the Doctorate of Education program. Three of the researchers were faculty members who serve on dissertation committees and one researcher was a student in her last semester in the EdD Program. The graduating student who participated in the research had her dissertation examined in the study, also making her a subject of the research. Each of the contributing researchers was selected due to his/her unique perspective of the program and the dissertation. The Chair of Advanced Professional Programs in the College of Education was selected due to her extensive history and expertise with the university. She had been with the university for more than 20 years at the time of the study. She also helped in the development of the Doctorate of Education program. She currently serves as a Doctorate of Education program dissertation chair and committee member. The Chair of the Doctorate of Education program has only been with the university for two years however he provides the leadership and guidance for the program and serves as a dissertation chair and committee member. The faculty member in the Doctorate of Education program has been with the university for seven years and teaches the final course in the program in which the students create presentations and publications from their dissertations. She also serves as a dissertation chair and committee member. Finally, the student in the Doctorate of Education program was in her final course of the program when the study was conducted. She has since successfully graduated. She serves as a leader of school psychology in a regional educational service center administer by the state education agency. Data Source The data source in this study included the 19 dissertations completed by the graduates of the education doctorate program in 2017 and 2018. The dissertations followed the same program- required dissertation template (a faculty-developed APA-formatted list of the appropriate dissertation chapter sections and brief descriptions of these sections), and ranged in page length from 88 to 220 pages with an average length of 138.25 pages. Of the 19 dissertations included in the study, there were six qualitative dissertations, nine quantitative dissertations, and four mixed methods dissertations. The authors of the 19 dissertations analyzed in this study were from a variety of professional roles including principal, professor, teacher, school psychologist, school improvement consult, school counselor, superintendent, director of secondary education, and gifted coordinator (See Table 1). The majority of the analyzed data was located in the Purpose of the Study in Chapter One and the Results and Conclusions in Chapter Five. The dissertations were completed prior to the education doctorate program’s initiation into CPED. Data Analysis To begin the document analysis, a pilot study was conducted in which the researchers independently reviewed the student- researcher’s dissertations and completed a dissertation summary chart for each dissertation. The dissertation summary chart developed by the researchers included the following information: title, author, professional role, problem origin (felt difficulty or real- world dilemma) (Ma et al., 2017), problem statement, impact, and alignment to the CPED six guiding principles. Through the pilot study, the researchers learned that they were all in agreement in respect to the problem statement and impact. In part, the researchers wanted to determine the degree to which their students’ work conformed to the CPED six guiding principles prior to participation in CPED. However, it was determined that further clarification on the CPED six guiding principles was needed before they could reach agreement and include the six principles on the summary chart. Therefore, the dissertation summary chart was revised to include only the title, author, professional role, problem origin, problem statement, and impact. In addition, the researchers learned that it was important for them to include the rationale for the problem of origin determination. Thus, an additional category title problem origin rationale was added to the dissertation summary chart. 5 Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 4 No. 1 (2019) DOI 10.5195/ie.2019.85 5 Table 1: Dissertation Title, Author, and Author Professional Role Title Author Professional Role Teacher Perceptions of the Ceiling Effect with Gifted Students and the Impact on Teacher Value-Added Scores and Teacher Evaluation Billings Gifted Coordinator Parental Involvement in the Digital Age:
Examining Parental Access to Student Web Portals in Grades 7-12 Bocian Administrator The Effectiveness of the Ohio School Leadership Institute Denecker Superintendent Effects of Teacher Evaluation on Teacher Job Satisfaction in Ohio Downing Educational Supervisor A Study of Pre-Service Teacher Efficacy During a Phonics Field Experience Eichner Adjunct Instructor Does School Discipline Style Make a Difference? Haselman Superintendent Identity Crisis: A Comparison of Stakeholder Perceptions Regarding K-12 Educational Mission Hlasko Superintendent The Impact on Teaching and Learning of the One-to-Ohio Laptop Initiative at the Ayersville Local Schools Hug Administrator Thinking About Teaching: Does Mentoring a Student Teacher Cause a Cooperating Teacher to be More Reflective? Laverick University Faculty Gender Differences in the Homework Preferences of Students with Low Self-Regulation Lee Teacher Relationships of Stakeholder Perceptions of School Climate Morse Teacher Impact of Academic and Nonacademic Support Structures on Third Grade Reading Achievement Peugeot Special Education Administrator College Aspirations to Completed Applications: A Study of Intentional High School Practices Designed to Increase Post-Secondary Enrollment Riepenhoff Administrator Examining the Efficacy of Multiple Intelligence-Based Pedagogical Approaches on Males in Elementary Mathematics Roush Teacher Predicting Ohio Principals’ Intentions and Practices toward State Evaluation-Based Professional Growth Plans Using the Theory of Planned Behavior Schooler Principal School-Sponsored Athletic Participation and the Impact on Student Achievement at the High School Level Selgo Principal The Impact of Social Networks on the Coming-Out Process for Lesbian, Gay, and Bisexual Individuals Walters-Powell University Faculty Factors Influencing Reading Growth in Online K-2 Students Wotring Principal Once the pilot study was complete, the researchers independently reviewed the 19 dissertations and completed the dissertation summary chart for each of them. As the Ma et al. (2017) study revealed, Bowen (2009) recommends document analysis include a combination of content analysis and thematic analysis. The dissertation summary chart included the necessary categories by which the dissertations underwent an initial content analysis as the information in the dissertation summary chart was organized by the research questions. The researchers then conducted a more complex thematic analysis for each category aligned with the research questions to determine patterns of emerging codes and themes in the data (Bowen, 2009). RESULTS Research Question One To answer the first research question (How do education doctorate students derive their problems of practice?), an initial analysis of the dissertations was conducted, and the researchers determined that there was interpretation in understanding a problem of practice as a felt difficulty or a real world dilemma. According to Ma et al. (2007), a felt difficulty is a “deep concern or dissatisfaction upon which the practitioner felt an urge to take action” (p. 15), whereas a real world dilemma is a: …situation an educational practitioner faced where he/she had to navigate existing tensions to improve their work… a definitive problem students set out to positively impact or resolve, these were problems situated in larger educational policy and procedures that created dilemmas for educators in their daily work. (p.16) In the current study, an example of a problem of practice derived from a felt difficulty was a doctoral student working as a teacher educator who noticed how a phonics field experience for teacher candidates impacted their self-efficacy. Another example was a doctoral student employed as a teacher who identified a difference between girls and boys in their homework preferences. An example of a problem of practice derived from a real world dilemma was a doctoral student working as a gifted coordinator who witnessed the impact of the ceiling effect on teacher and students. Despite the variability in interpretation in understanding a problem of practice as a real world dilemma or a felt difficulty by the researchers, there was a majority consensus for more than half of the dissertations: four real world dilemma problems of practice and six felt difficulty problems of practice (see Table 2). The four real world dilemma problems of practice included the dissertations on gifted students, school discipline, one-to-one laptops, and mentor teacher reflection. The six felt difficulty problems of practice included the dissertations on pre-service teacher efficacy, homework and gender, elementary males and math, athletic participation, social media and LGB students, and elementary online reading. There was only one dissertation in which all four researchers agreed on the type of problem of practice, felt difficulty: the dissertation on pre-service teacher efficacy. The researchers learned that the graduate students did not use the term problem of practice to identify their dissertation studies despite the fact that they were problems of practice. The graduate students also did not identify their studies as real world dilemmas or felt difficulty. However, the Doctor of Education program faculty only recently joined CPED and became familiar with this terminology. Therefore, it was not expected that students would identify their work using these terms. However, the researchers wanted to assess their Gillham, Williams, Rife, and Parker Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 4 No. 1 (2019) DOI 10.5195/ie.2019.85 6 students’ work to determine if these concepts already existed prior to the program’s involvement with CPED. The researchers’ perceptions of the problem of practices as a real world dilemma or a felt difficulty appeared to be impacted by the professional role of the researcher. For example, the Chair of Advanced Professional Programs in the College of Education has a background, education, and expertise in the hard sciences. Her more positivist paradigm, with its conception of objective, knowable truth, perhaps influenced her perception that 11 out of the 19 dissertations were real world dilemma problems of practice. The faculty member in the Doctorate of Education program has a background, education, and expertise in the social sciences. Her more postmodern paradigm, with a conception of truth as being relative and individually constructed, perhaps influenced her perception that 16 out of the 19 dissertations were felt difficulty problems of practice. The Chair of the Doctor of Education program was a bit more mixed in his perceptions of the problems of practice as was the student in the Doctor of Education program. Although the concepts of real world dilemmas and felt difficulties appeared to be well-defined, the lens of each researcher tended to guide their interpretation of these concepts in practice. Regardless, the researchers learned through the document analysis that the education doctorate students derived their problem of practice predominantly through felt difficulties more so than real world dilemmas. Research Question Two The initial analysis of the second research question (What is the nature of the problems of practice that the students have studied?), revealed the nature of the problems of practice to be somewhat diverse (see Table 2). The majority of the problems of practice investigated appeared to be guided by the graduate student’s professional role. For example, the three graduate students whose professional role was a teacher studied aspects of the classroom: homework preferences, school climate, and pedagogical approaches. The three graduate students whose professional role was in post-secondary education researched undergraduate students: pre-service teachers and lesbian, gay, and bisexual (LGB) students. The majority of the graduate students whose professional role was a superintendent or administrator studied a variety of problems of practice related to educational leadership. In the dissertation titled, The Effectiveness of the Ohio School Leadership Institute, the doctoral student clearly identified his professional role and how it informed the problem of practice he researched: This researcher served in administration as a high school principal for eight years and has spent nearly a decade as a school district superintendent. During the 2012-2013 academic year, this researcher had the opportunity of being selected as one of 30 superintendents across the state of Ohio to participate in the Ohio School Leadership Institute, sponsored (and funded) by the Buckeye Association of School Administrators (BASA). Many of the graduate students acknowledged not only their professional role and how it guided their problem of practice study but also how their professional role could bias the study. This was addressed by most graduate students in their section on researcher bias in the first chapter. For example, in the dissertation titled, The Impact of Social Networks on the Coming-Out Process for Lesbian, Gay, and Bisexual Individuals, the graduate student explained her researcher bias as it relates to her professional roles: The investigator has a background in social work, which provides a deeper awareness of the long-term issues created by a lack of support from family, peers, education, spirituality, and great community on an individual that already is considered at-risk… The researcher identifies as a Professor of Social Work and advisor to the Gay Straight Alliance, both of which provide a vast amount of exposure to working with these individuals, which could then lead to unintentional stereotyping of participants. This statement of researcher bias is representative of many of the graduate students’ recognition and acknowledgment of their professional role and the impact on the problem of practice dissertation. However, researcher bias was not a factor in the analysis for this study. Research Question Three The third research question that guided this study (What are the reported impacts the study of problems of practice has on doctoral students’ local contexts?) directly connected to the scholarly significance of this study. Through this study, the researchers found the type of problem of practice researched through the education doctorate dissertation contributed to the perceived impact on the local context of the author. However, the authors of the dissertations reported only the perceived impacts on the local context as opposed to the reality of impact on the local context. Doctoral students were not required to report an implementation plan or to file a follow-up impact report. Rather, the researchers analyzed the potential impact as it were framed in the dissertation’s implications and recommendations. The perceived reported impacts were on gifted students, students in P-12 schools (7), superintendents and administrators, teachers (2), teacher candidates, educational stakeholders, teachers and students, mentor teachers, schools, underrepresented students, and LGB students. The researchers noticed a lack of specificity in the dissertations related to how the authors’ perceived their dissertation study would impact the local context. For example, in the study on principals’ intentions and practices related to evaluation-based professional development growth plans, the author wrote “the significance of this study is that it dispels the widely held myth that educators already know how to help teachers improve; and the research based for what actually helps teachers improve is extremely thin.” More depth and description was needed by the dissertation author to determine how she perceived the significance of her research as stated above impacts the field of teacher evaluation in both a local context and in terms of generalizability. Problems of Practice: A Document Analysis of Education Doctorate Dissertations Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 4 No. 1 (2019) DOI 10.5195/ie.2019.85 7 Table 2. Perception of Dissertation Problem of Practice by Researcher Topic Chair of Ad Pro Chair of Doc Faculty Member Student Gifted Students Real World Felt Difficulty Real World Real World Parental Involvement Real World Felt Difficulty Felt Difficulty Real World Superintendent Leadership PD Real World Felt Difficulty Felt Difficulty Real World Teacher Evaluation Felt Difficulty Real World Felt Difficulty Real World Pre-Service Teacher Efficacy Felt Difficulty Felt Difficulty Felt Difficulty Felt Difficulty School Discipline Real World Real World Felt Difficulty Real World Educational Mission Felt Difficulty Real World Felt Difficulty Real World One-to-One Laptops Real World Real World Real World Felt Difficulty Mentor Teacher Reflection Real World Real World Real World Felt Difficulty Homework and Gender Real World Felt Difficulty Felt Difficulty Felt Difficulty School Climate Real World Felt Difficulty Felt Difficulty Real World Support Structures Felt Difficulty Real World Felt Difficulty Real World Post-Secondary Enrollment Felt Difficulty Real World Felt Difficulty Real World Elementary Males and Math Real World Felt Difficulty Felt Difficulty Felt Difficulty Teacher Evaluation PD Felt Difficulty Real World Felt Difficulty Real World Athletic Participation Real World Felt Difficulty Felt Difficulty Felt Difficulty Counselor Evaluation Felt Difficulty Real World Felt Difficulty Real World Social Media and LGBT Real World Felt Difficulty Felt Difficulty Felt Difficulty Elementary Online Reading Felt Difficulty Felt Difficulty Felt Difficulty Real World DISCUSSION, IMPLICATIONS AND RECOMMENDATIONS The purpose of this study was to learn how education doctorate students, who were predominately current educators in the P-12 schools, created the problems of practice researched in their dissertations and the potential impact of their research on their local contexts. In respect to the first research question, the researchers’ learned through the document analysis that the education doctorate students derived their problems of practice predominantly through felt difficulties more so than real world dilemmas. However, the researchers determined that there was immense interpretation in understanding whether a problem of practice is either a real world dilemma or a felt difficulty. There was only one dissertation that all four researchers agreed on the type of problem of practice: the dissertation on pre-service teacher efficacy. The initial findings indicate a need for further refinement of the concept of a problem of practice and how education doctorate programs and their candidates employ the concept of a problem of practice in their dissertations studies and how this impacts local contexts. Ma et al. (2017) stated: …the students’ work settings serve as a laboratory of practice (Shulman 2006) and course assignments require students to apply what they learn in coursework immediately in their local contexts, often collecting and analyzing data to reflect on the ways what they are learning in coursework translates into their practice. (p. 11) In addition, these researchers described how the students in their program are required to create a dissertation prospectus that outlines how the student’s dissertation research addresses a problem of practice in their local context. The intentional identification and purposeful inclusion of the graduate students’ local context as a setting to investigate a problem of practice throughout the program coursework and key assessments is necessary to help them understand their dissertations as problems of practice to take action in their local contexts. The problems of practice revealed through the document analysis of the dissertations in this study revealed immense variety (see Table 2). In their study, Ma et al. (2017) identified three clear themes that emerged from the dissertation problems of practice: 1) supporting marginalized students or group, 2) increasing the quality of educator professional development, and 3) supporting novices’ entry into the profession. In the current study, the researchers were unable to determine clear themes in the problems of practice. Instead, the analyses of the dissertation revealed a diverse study of problems of practice in the field of education. With the inclusion of identification and purposeful inclusion of the graduate students’ local context as a setting to investigate a problem of practice throughout the program coursework and key assessments, the researchers hypothesize this could perhaps narrow the focus of the problems of practice. This is a potential investigation for future research. Finally, the authors of the dissertations in this study reported only the perceived impacts on the local context as opposed to the reality of impact on the local context. The researchers noticed a lack of specificity in the dissertations related to how the authors’ perceived their dissertation study would impact the local context. Doctoral students were not required to report an implementation plan or to file a follow-up impact report. Rather, the researchers analyzed the potential impact as it were framed in the dissertation’s implications and recommendations. Ma et al. (2017) reported five themes of impact based on their dissertation document analysis: 1) advancing/changing researchers’ practice and beliefs, 2) promoting communication and collaboration, 3) strengthening/informing policy, 4) supporting student learning, and 5) fostering colleagues’ learning. With the inclusion of identification and purposeful inclusion of the graduate students’ local context as a setting to investigate a problem of practice throughout the program coursework and key assessments, the researchers hypothesize this could perhaps provide a much stronger understanding of how the graduate students dissertation study will impact their local context. This is another potential investigation for future research. Gillham, Williams, Rife, and Parker Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 4 No. 1 (2019) DOI 10.5195/ie.2019.85 8 In conclusion, the researchers learned through the document analysis of the education doctorate dissertations that while the graduate students in the program study problems of practice, a much greater emphasis needs to be placed on the four principles identified by Ma et al. (2017): 1. Problems of practice are deeply embedded in the students’ professional practice or context. 2. Problems of practice emanate from felt difficulties and real world dilemmas students face as they work as educational practitioners. 3. Problems of practice align with contemporary, critical issues in education explicated in the literature, such as creating more equitable schooling experiences for all children. 4. Problems of practice hold personal significance for the students’ developing professional identity as a practitioner scholar (p. 23). We look forward to further research on how these principles will guide our program and students in the true search for solutions to the problems of practice they encounter daily in their lived experiences as practitioner scholars. Specifically, the researchers look forward to learning more about CPED’s six guiding principles and how to incorporate them into our program. The researchers hope to develop signature pedagogies that will empower scholarly- practitioners who can make an impact on their local context. REFERENCES Anderson, D. G. (1983). Differentiation of the Ed.D. and Ph.D. in education. Journal of Teacher Education, 34(3), 55–58. Andrews, R., & Grogan, M. (2005, Spring). Form should follow function: Removing the EdD dissertation from the Ph.D. straight jacket. UCEA Review, pp. 10-12. Archbald, D. (2008). Research versus problem solving for the education leadership doctoral thesis: Implications for form and function. Educational Administration Quarterly, 44(5), 704–739. Argyris, C. (1982). Reasoning, learning, and action. San Francisco: Jossey- Bass. Belzer, A., & Ryan, S. (2013). Defining the problem of practice dissertation: Where’s the practice, what’s the problem? Planning and Changing, 44(3), 195-207. Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative Research Journal, 9(2), 27-40. Bowen, W.G., & Rudenstine, N.L. (1992). In pursuit of the PhD. Princeton, NJ: Princeton University Press. Carnegie Project on the Education Doctorate (CPED). (2017). Beyond a definition: Designing and specifying dissertation in practice (DiP) models. Retrieved from http://www.cpedinitiative.org/ Carnegie Project on the Education Doctorate (n.d.a). About us. Retrieved from http://www.cpedinitiative.org/ Carnegie Project on the Education Doctorate (n.d.b). The framework. Retrieved from https://www.cpedinitiative.org/page/framework City, E. A., Elmore, R. F., Fiarman, S. E., & Teitel, L. (2009). Instructional rounds in education: A network approach to improving learning and teaching. Cambridge, MA: Harvard Education Press. Clifford, G. J., & Guthrie, J. W. (1988). Ed school: A brief for a professional education. Chicago: University of Chicago Press. Dagenais, C., Lysenko, L., Abrami, P. C., Bernard, R. M., Ramde, J., & Janosz, M. (2012). Use of research-based information by school practitioners and determinants of use: a review of empirical research. Evidence & Policy: A Journal of Research, Debate and Practice, 8(3). Policy Press. https://doi.org/10.1332/174426412X654031 Deering, T. E. (1998). Eliminating the doctor of education degree: It’s the right thing to do. Educational Forum, 62(3), 243-248. Furman, G. (2012). Social justice leadership as praxis: Developing capacities through preparation programs. Educational Administration Quarterly, 48(2), 191-229. Gilbert, R., J. Balatti, P. Turner, & Whitehouse, H. (2004). The generic skills debate in research higher degrees. Higher Education Research and Development, 23(3), 376-388. Golde, C. M. (2007). Signature pedagogies in doctoral education: Are they adaptable for the preparation of education researchers. Educational Researcher, 36(6), 344–351. https://doi- org.ezproxy.findlay.edu/10.3102/0013189X07308301 Golde, C. M., & Walker, G. E. (2006). Envisioning the future of doctoral education: Preparing stewards of the discipline. San Francisco, CA: Jossey-Bass. Grogan, M., & Andrews, R. (2002, April). Defining preparation and professional development for the future. Educational Administration Quarterly, 38(2), 233-256. Hochbein, C., & Perry, J. A. (2013). The role of research in the professional doctorate. Planning and Changing, 44(3), 181–194. Hoffman, R. L., & Perry, J. A. (2016). The CPED framework: Tools for change. In J. S. Perry (Ed.) The EdD and the scholarly practitioner: The CPED path (pp. 13-25). Charlotte, NC: Information Age Publishing. Kot, F.C., & Hendel, D. D. (2012). Emergence and growth of professional doctorates in the United States, United Kingdom, Canada and Australia: A comparative analysis. Studies in Higher Education, 37(3), 345-364.
 Kumar, S., & Dawson, K. (2012). Exploring the impact of a professional practice education doctorate in educational environments, Studies in Continuing Education, 35(2), 165-178. DOI: 10.1080/0158037X.2012.736380 Levine, A. (2005). Educating school leaders. New York: Columbia University Teachers College. Ma, V. W., Dana, N. F., & Adams, A., & Kennedy B. (2017). Understanding the problems of practice: An analysis of professional practice Ed.D. dissertations. Paper presented at the 2017 AERA Annual Meeting. San Antonio, TX. Malen, B., & Prestine, N. (2005, Spring). The case for revitalizing the dissertation. UCEA Review, pp. 7-9.
 Maxwell, T. W. K. (2009). Producing the professional doctorate: The portfolio as a legitimate alternative to the dissertation. Innovations in Education & Teaching International, 46(2), 135–145. Mowbray, S., & C. Halse. 2010. The purpose of the PhD: Theorizing the skills acquired by students. Higher Education Research and Development, 29(6), 653-664. Murphy, J., & Vriesenga, M. (2005). Developing professionally anchored dissertations. School Leadership Review, 1(1), 33-57.
 National Science Foundation (2018). National Center for Science and Engineering Statistics: Survey of Earned Doctorates. Retrieved from https://www.nsf.gov Nelson, J. K., & Coorough, C. (1994). Content analysis of the PhD versus EdD dissertation. The Journal of Experimental Education, 62(2), 158-168. Onwuegbuzie, A. J., & Dickinson, W. (2006). Mixed methods research and action research: A framework for the development of preservice and inservice teachers. Retrieved from http://www.unco.edu/ae- extra/2007/6/Onwuegbuzie.html Osguthorpe, R. T., & Wong, M. J. (1993). The Ph.D. versus the Ed.D: Time for a decision. Innovative Higher Education, 18(1), 47-63.
 Perry, J. A. (2012). What history reveals about the education doctorate. In Latta, M. M., & Wunder, S. (Eds.) Placing practitioner knowledge at the center of teacher education: Rethinking the policy and practice of the education doctorate. Somerset, MA: Information Age Publishing. Shulman, L. S., Golde, C. M., Bueschel, A. C., & Garabedian, K. J. (2006). Reclaiming education’s doctorates: A critique and a proposal. Educational Researcher, 35(3), 25-32. Storey, V. A., Caskey, M. M., Hesbol, K. A., Marshall, J. E., Maughan, B., & Dolan, A. W. (2015). Examining EdD dissertations in practice: The Carnegie Project on the Education Doctorate. International HETL Review, 5(2), Retrieved from https://www.hetl.org/examining-edd- dissertations-in-practice-the-carnegie-project-on-the-education- doctorate/ http://www.cpedinitiative.org/ http://www.cpedinitiative.org/ https://www.cpedinitiative.org/page/framework https://doi.org/10.1332/174426412X654031 https://doi-org.ezproxy.findlay.edu/10.3102/0013189X07308301 https://doi-org.ezproxy.findlay.edu/10.3102/0013189X07308301 https://www.nsf.gov/ http://www.unco.edu/ae-extra/2007/6/Onwuegbuzie.html http://www.unco.edu/ae-extra/2007/6/Onwuegbuzie.html https://www.hetl.org/examining-edd-dissertations-in-practice-the-carnegie-project-on-the-education-doctorate/ https://www.hetl.org/examining-edd-dissertations-in-practice-the-carnegie-project-on-the-education-doctorate/ https://www.hetl.org/examining-edd-dissertations-in-practice-the-carnegie-project-on-the-education-doctorate/ Problems of Practice: A Document Analysis of Education Doctorate Dissertations Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 4 No. 1 (2019) DOI 10.5195/ie.2019.85 9 Tennant, M. (2004). Doctoring the knowledge worker. Studies in Continuing Education, 26(3), 431-441. Toma, J. D. (2002, November). Legitimacy, differentiation, and the promise of the EdD in higher education. Paper presented at annual meeting of the Association for the Study of Higher Education, Sacramento, CA.
 Townsend, B. (2002, November 21-24). Rethinking the EdD or what’s in a name? Paper prepared for the annual meeting of the Association for the Study of Higher Education, Sacramento, CA.
 Walker, D. W. & Haley-Mize, S. (2012). Content analysis of PhD and EdD dissertations in special education. Teacher Education and Special Education, 35(3), 202-211. DOI: 10.1177/0888406411431168 Wergin, J. F. (2011). Rebooting the Ed.D.. Harvard Educational Review, 81(1), 119–139.
 Willis, J., Inman, D., & Valenti, R. (2010). Completing a professional practice dissertation: A guide for doctoral students and faculty. Charlotte, NC: Information Age. Zambo, D. (2011). Action research as signature pedagogy in an education doctorate program: The reality and hope. Innovative Higher Education, 36(4), 261–271. Zambo, R., Zambo, D., Buss, R. R., Perry, J. A., Williams, T. R. (2014). Seven years after the call: Students’ and graduates’ perceptions of the re- envisioned Ed.D.. Innovative Higher Education, 39(1), 123-137.