Impacting Education: Journal on Transforming Professional Practice New articles in this journal are licensed under a Creative Commons Attribution 4.0 United States License. This journal is published by the University Library System of the University of Pittsburgh as part of its D-Scribe Digital Publishing Program and is cosponsored by the University of Pittsburgh Press. 1 This journal is supported by the Carnegie Project on the Education Doctorate: A Knowledge Forum on the EdD (CPED) cpedinitiative.org impactinged.pitt.edu ISSN 2472-5889 (online) Vol. 6, No. 4 (2021) DOI 10.5195/ie.2021.144 Using Evidence to Frame Problems of Practice Lesley F. Leach Tarleton State University leach@tarleton.edu Credence Baker Tarleton State University cbaker@tarleton.edu Catherine G. Leamons Tarleton State University catherine.leamons@go.tarleton.edu Phillis Bunch Tarleton State University sw_bunch@tarleton.edu Jesse Brock Tarleton State University sw_jbrock@tarleton.edu ABSTRACT Improvement initiatives crafted based on well-understood problems of practice often stand the greatest chance of leading to sustainable educational improvements. Framing problems of practice using multiple modes of evidence is advisable to fully understand the system of root causes of the problem and its stakeholders. In this study, we used the document analysis method to investigate the types of evidence (e.g., literature, anecdotal, secondary data) that students used to frame problems of practice in EdD dissertations in practice within CPED consortium member institutions (N=53). Results suggest that students predominantly use literature to frame problems of practice with fewer using primary and secondary data. KEYWORDS problem of practice, evidence, framing, root cause analysis Decades of reform efforts have yet to produce long-term, sustainable solutions to the issues plaguing the U.S. educational system. Too often, in efforts to create quick change, educational leaders apply interventions that worked in other environments without considering the differences between the two environments and how those differences could negatively impact the intervention’s success (Bryk et al., 2016). Such interventions perpetuate the cycle prevalent in education in which initiatives are adopted but quickly abandoned because they are proven ineffective or are not provided enough time to show the desired effect (Rohanna, 2017). Some have argued that most large-scale initiatives fail due to limited understandings of the very problems the initiatives were undertaken to solve or improve (Bryk et al., 2016). The understanding of the problem could be limited by a lack of data or a lack of understanding about the larger systems in which the problem is situated. Take, for instance, the pervasive issue of mathematics achievement that exists at all levels of education in the U.S.—from students’ low Algebra passage rates in K-12 to the difficulty community colleges and higher education institutions have moving students beyond developmental mathematics. A K-12 leader would likely first detect the issue from low scores on standardized tests to which the leader might respond by offering tutoring sessions to students. But if the leader investigated further, she would likely find that the problem stemmed from a wide set of factors including teachers’ lack of training to effectively teach mathematics, students’ language barriers, and curricula focused on rote memorization of facts rather than problem-solving skills. Put simply, low test scores were symptoms of the problem; there was likely a system of root causes. And it is likely the system of root causes that should guide potential improvements to create meaningful and sustainable solutions. CARNEGIE PROJECT ON THE EDUCATION DOCTORATE Helping to lead the charge of defining and improving upon educational problems are institutions within the Carnegie Project on the Education Doctorate (CPED) consortium. Currently, there are http://www.library.pitt.edu/ http://www.pitt.edu/ http://www.library.pitt.edu/articles/digpubtype/index.html http://www.library.pitt.edu/articles/digpubtype/index.html http://upress.pitt.edu/ http://creativecommons.org/licenses/by-nc-nd/3.0/us/ http://cpedinitiative.org/ mailto:leach@tarleton.edu mailto:cbaker@tarleton.edu mailto:catherine.leamons@go.tarleton.edu mailto:sw_bunch@tarleton.edu mailto:sw_jbrock@tarleton.edu https://orcid.org/0000-0001-9040-7207 https://orcid.org/0000-0002-7814-2135 https://orcid.org/0000-0003-3326-2261 https://orcid.org/0000-0002-2943-9717 Leach, et al. Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 6, No. 4 (2021) DOI 10.5195/ie.2021.144 2 over 100 CPED member institutions (CPED, n.d.-a) who, as “a network of ‘peer-to-peer’ faculty members and leaders” (Perry & Imig, 2016, para. 9), strive to better prepare EdD programs to guide and produce leaders in education. According to Perry (2013), CPED is the first “action-oriented effort” aimed at reframing, redefining, and restructuring the EdD (Perry, 2013, p. 115). Through intellectual exercises and discussions at meetings, CPED members collaborated to define the EdD as a degree that “prepares educators for the application of appropriate and specific practices, the generation of new knowledge, and for the stewardship of the profession” (CPED, n.d.-a, para. 6). CPED member institutions created a framework (CPED, n.d. - c) that EdD programs could use to design their programs in ways that honored their particular student constituencies and local educational communities. CPED also created a set of design concepts that member institutions could use in their EdD programs to further the development of scholar practitioners. The design concepts include, among others, a signature pedagogy that is the hallmark pedagogical process particular to that program, similar to “rounds” in medical education or the “Socratic method” in legal education. For the EdD, graduates use “inquiry as practice” (CPED, n.d.-c, para. 1) to identify and improve upon educational issues in their organizations, and the dissertation in practice. Dissertation in Practice The cornerstone of CPED-influenced EdDs is the dissertation in practice (DiP), which is similar to a traditional dissertation in some ways but markedly different in others. Both traditional dissertations and dissertations in practice involve investigations of educational phenomena with extensive reviews of literature; however, the main difference between the two is that the dissertation in practice centers on an identified localized problem of practice rather than gaps in the research literature (Storey et al., 2015). And it is the investigation of improvements related to the problem of practice that often defines a DiP. Completed DiPs serve as evidence that EdD graduates possess the abilities, skills, and knowledge to “analyze problems of practice and use multiple frames to develop meaningful solutions” (CPED, n.d.-a, para. 6) while in EdD programs, but also in their current and future practice as educational leaders. Problems of Practice CPED defines a problem of practice as “a persistent, contextualized, and specific issue embedded in the work of a professional practitioner the addressing of which has the potential to result in improved understanding, experience, and outcomes” (CPED, n.d.-c, para. 1). Decomposition of the definition requires consideration of the three phrases within it. A persistent problem is one that happens more than once and likely for more than a short time period, with the timeframe relative based on the nature of the problem. A problem is contextualized when it is situated within a particular setting with particular characteristics. Finally, a problem is considered a “specific issue embedded in the work of a professional practitioner, the addressing of which has the potential to result in improved understanding, experience, and outcomes” (CPED, n.d.-c, para. 1) when it is one that is clearly localized to a practitioner’s work and in need of improvement. Mintrop (2016) furthered the CPED PoP definition by stressing the urgency of the solution to the localized issue. By virtue of the individuals and the types of issues one sees in education, PoPs are often ill-structured or ill-defined, meaning that they are complex and lack clear-cut solutions (Archbald, 2014; Copland, 2000; Jonassen, 2000; Timperley & Robinson, 1998). Educational problems do not often present themselves ready for solution; rather, full understanding of a problem is generated through the collective examination of multiple problematic situations (Schön, 1983). With the assistance of Mintrop (2016), we define what it means to identify, define, and frame a problem of practice below; these steps are necessary to gain a full understanding of a PoP and its system of root causes. Note that while the words identify, define, and frame may have general meanings elsewhere, the meanings related to problems of practice are quite specific to those offered below. The first step toward a full understanding of a PoP is to identify or select an issue that is an urgent need of an organization (Bryk et al., 2016; Mintrop, 2016). One might think of this as conception of the idea that completion of the next two steps in the process will further explicate. The process at which one comes to identification is likely different for various individuals and organizations but is often guided by organizational strategic plans or goals situated within the individual’s particular sphere of influence. Once an issue has been identified as potential problem of practice, it requires further definition. To define the problem, individuals tap prior knowledge and experience to consider the symptoms or instances that initially made the issue appear problematic (Mintrop, 2016). This has been characterized as the gap between the current state and the ideal (Archbald, 2014; Mintrop, 2016). Gaps represent comparisons such as discrepancies between one’s organization and another (e.g., institution A has lower scores than institution B), between an organization over time (e.g., institution A’s scores have decreased over the last 5 years), or between an organization and its targets (institution A’s scores did not meet the target) (Archbald, 2014). It is the justification of these gaps as problems, contends Archbald (2014), that defines the problem of practice and situates it as an issue for study and potential improvement. Framing the PoP demands consideration of the broader construction of the issue, including the myriad factors that likely impact it and stakeholders who influence it. Closely related to problem definition, framing of the problem goes one step further in considering the potential causes of the issue as well as individual assumptions and lenses that may influence the development of the frame (Schön & Rein, 1995). Investigators have empirically examined the nature and content of PoPs (see e.g., Gillham et al.,2019; Ma et al., 2018; Storey et al., 2015), but questions still remain about how to thoroughly frame a PoP. Naturally, scholars often turn to the literature for prior studies and theories that may inform the PoP (Archbald, 2014; Belzer & Ryan, 2013; Mintrop, 2016). But when considering the contextual nature of problems, the literature may only serve to help situate the identified issue within the broader context and may not be enough to provide information on the problem in the local context. Literature can help clarify the ideal state, but the most convincing problem definitions also include practitioner and stakeholder input through multiple modes of evidence, including secondary, primary, and/or anecdotal data in addition to relevant literature (Archbald, 2014). Mintrop (2016) also advocated framing problems using multiple modes of evidence, but through an iterative design-based process. Using Evidence to Frame Problems of Practice Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 6, No. 4 (2021) DOI 10.5195/ie.2021.144 3 The process begins with an initial framing of the PoP and identification of potential improvement initiatives via broad problem statements. The broad statements are further refined as evidence is gathered and assimilated, with a portion of evidence derived as the result of an empirical needs assessment of the organization (Mintrop, 2016). The improvement science (IS) methodology advances a systems analysis to build a robust understanding of problems and avoid “solutionitis” (Bryk et al, 2016, p. 197) where solutions are enacted before the problem is fully understood. In contrast to experimental science that seeks to minimize variation, IS embraces the idea that “variation in implementation and setting are important sources of information” (Lewis, 2015, p. 55) in solving problems. IS advances a causal system analysis to investigate root causes of problems that might otherwise be obscured by their symptoms. From the causal system analysis develops a solution system that addresses the particular influence of each identified system on the problem and works to improve upon it. The solution is then implemented in iterative plan-do-study-act (PDSA) cycles whereby the intervention is planned and enacted, results are studied, changes are made, and the implementation cycle begins again with the revised plan. Iterative cycles of inquiry provide a mechanism to “[learn] fast to implement well” (Bryk et al., 2016, p. 7) in the aim of continuous improvement. NEED FOR EVIDENCE IN POST-TRUTH ERA The need for evidence to substantiate PoPs is arguably more relevant in the current post-truth era where “facts are less influential in shaping public opinion than appeals to emotion and personal belief” (Oxford Languages, 2018, para. 1). Because they invariably do not uncover the true nature of PoPs, issues framed by personal agenda, emotion, or even anecdotal evidence likely prompt misguided reforms that stand little to no chance of creating long-term sustainable change in education. Evidence is needed to substantiate PoPs in education, and that evidence needs to be of sufficient rigor and quality to withstand the “contempt for facts” (Arendt, 1957, p. 350) that exists in the current post-truth society. STUDY PURPOSE Currently, as the dissertation in practice continues to develop in redesigned EdD programs, it is unknown to what extent evidence is used to frame problems of practice within dissertations in practice. Thus, the purpose of this study was to explore the use of evidence in framing PoPs within published dissertations in practice. We sought to understand the nature of PoPs, whether they met the CPED definition of a PoP, what types of evidence and methods were used to frame the identified PoPs, and to what degree particular methods were used to frame PoPs. Note that the study was limited to the framing of problems of practice rather resulting improvements or proposed solutions. METHOD We used the document analysis method (Bowen, 2009) to gain a more thorough understanding of the evidence used to frame PoPs as have other authors that studied PoPs (see e.g., Gillham, Williams, Rife, & Parker, 2019; Ma, et al., 2018). According to Bowen (2009), “Document analysis is a systematic procedure for reviewing or evaluating documents - both printed and electronic” (p. 27) for which documents are reviewed to extract data. Data are then analyzed and interpreted to make meaning and gain understanding regarding the research question(s) or purpose (Bowen, 2009). The primary data source comprised dissertations in practice (N=53, from original N=55 with 2 deletions for broken links) that were publicly available through the CPED website (CPED, n.d-b.). Note that the dissertations were likely not a representative sample of dissertations in practice from CPED member institutions, so may not be generalizable to the larger population of dissertations in practice as a result. The dissertations linked from the CPED website comprise the only current centralized source of dissertations in practice known to the authors as of this writing. Limitations notwithstanding, data should still offer interesting results that can contribute to the discussion of how PoPs are and should be framed. Following the document analysis method, the following were coded for each dissertation: (a) the topic of each PoP, (b) whether the PoP was stated or implied, (c) whether the PoP met the three components of the PoP definition, (d) the evidence provided to frame the PoP, (e) the methodology used to frame the PoP, and (f) the dissertation’s institution. All three characteristics of the CPED definition were coded including whether the identified problem of practice was (a) “persistent”, (b) “contextualized”, and (c) “a specific issue embedded in the work of a professional practitioner” (CPED, n.d.-c, para. 1). For the purposes of coding, we considered evidence of the persistent nature of a PoP as at least one reference or sentence regarding the long-lasting nature of the PoP, with “long- lasting” being relative based on the nature of the PoP. Evidence of the contextualized nature of the PoP comprised at least one reference or sentence regarding the conditions within which the PoP was situated. Evidence that the work was embedded in the work of a professional practitioner comprised at least one statement that signified whether the topic was localized and practice-based. It was difficult to discern whether a PoP was embedded directly into the work of the author without directly knowing the author and their workplace, so whose work it was embedded within was disregarded. As a calibration exercise, the researchers reviewed a set of three dissertations together, and codes were discussed until all authors were in agreement. The remaining dissertations were each randomly assigned to two researchers to be coded individually. Any discrepancies were flagged and discussed between each pair until consensus was reached. Finally, the entire research team reviewed the spreadsheet together and tabulated results to gain an understanding of initial results followed by process of categorization and thematic analysis as suggested by Bowen (2009). Interrater agreement between the multiple coders strengthened the credibility of results as well as the fact that coding pairs were randomly assigned and varied for each dissertation in practice. Much as particular individual’s assumptions and lenses can impact the framing of a particular problem of practice, so too could they have impacted our data collection and analysis. The authors are faculty and students in a CPED-influenced EdD program and acknowledge potential biases that could have influenced the study results. Leach, et al. Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 6, No. 4 (2021) DOI 10.5195/ie.2021.144 4 RESULTS The coded data were analyzed descriptively. Frequencies were run to analyze how each PoP was characterized—specifically, was it explicitly characterized as a problem of practice, was it implied to be a problem of practice without the explicit language, or was it not stated or implied to be a problem of practice? Table 1 presents the results. The majority of dissertations (85%, which comprised 45 of N=53) involved a problem of practice that was stated or implied. But just 27% of the dissertations (14 of N=53) explicitly characterized PoPs as such. Fifty-eight percent (31 of N=53) offered evidence that implied that the identified issues were problems of practice but did not state it explicitly, and 15% (8 of N=53) were not considered to be PoPs and addressed gaps in the literature or other. Table 1 also presents the results of descriptive analysis of coding regarding evidence that the PoP met all components of the CPED definition—i.e., that the problem of practice was (a) “persistent”, (b) “contextualized”, and (c) “a specific issue embedded in the work of a professional practitioner” (CPED, n.d.-c, para. 1). The majority of PoPs (43, which comprised 81% of N=53) did not offer evidence to show that all components of the CPED definition of a PoP were met. Interestingly, of the dissertations that addressed a PoP (N=45), most dissertations included evidence of at least one component of the definition, with 78% (35 of N=45) providing evidence of the persistence of the PoP, 87% (39 of N=45) providing evidence that the PoP was contextualized, and 84% (38 of N=45) demonstrating that the issue was specific and within the work of a professional practitioner. However, only 19% of all dissertations (10 of N=53) provided evidence of all three components of the definition. We coded the evidence and methods used to frame PoPs within the sampled dissertations in practice; results are presented in Figure 1. PoPs overwhelmingly offered references to the literature as evidence of the problems of practice (41, which comprised 77% of N=53 dissertations). This particular result was perhaps not surprising given the emphasis of the literature on the theoretical, empirical, and practical bases on which problems may be framed. Fewer PoPs (23, which comprised 45% of N=53) offered practitioner expertise or anecdotal accounts as evidence of the PoPs. Data were presented in several cases - with more PoPs justified using secondary data (18, which comprised 34% of N=53) than primary data (17, which comprised 32% of N=53). Particularly compelling, however, is the fact that of all the dissertations in practice—even the ones that incorporated data as evidence of the PoP—just two dissertations (4% of N=53) employed a systematic methodology to frame the PoP. The two dissertations used empirical needs assessments to collect data as has been suggested by Mintrop (2016), however they lacked the systems analysis advanced in improvement science (Bryk et al., 2016). We examined the types of evidence offered for each of the PoPs by whether each met the CPED definition of a PoP; Figure 2 presents the results. The general pattern of results generally mirrored the results presented in Figure 1. However, the percentage of dissertations using primary (4, which comprised 40% of n=10) or secondary data (4, which comprised 40% of n=10) as evidence was a bit higher in the dissertations where the definition of the PoP was justified over those that did not (primary data – 13, which comprised 30% of n=43; secondary data – 13, which comprised 30% of n=43). Table 1. Percentages of Dissertations in Practice that Presented Evidence of All Components of the PoP Definition Note. The definition of a problem of practice (PoP) as “a persistent, contextualized, and specific issue embedded in the work of a professional practitioner” (CPED, n.d.-a) was used to classify dissertations into those that met the definition and those that did not. Percentages represent the number divided by the total number of dissertations (N=53) rounded to the nearest whole number; the rounded numbers may not add to 100% as a result. Figure 1. Proportions of Evidence Sources for Problem of Practice (N=53) . Note. Problems of practice were often framed using multiple sources of evidence, so percentages of the sources of evidence will add to greater than 100% PoP Definition Problem of Practice Characterization Did not meet Met Total Not characterized as PoP 8 (15%) 8 (15%) PoP implied but not stated 25 (47%) 6 (11%) 31 (58%) PoP explicitly stated 10 (19%) 4 (8%) 14 (27%) Total 43 (81%) 10 (19%) 53 (100%) 1 Using Evidence to Frame Problems of Practice Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 6, No. 4 (2021) DOI 10.5195/ie.2021.144 5 Figure 2. Proportions of Evidence Sources by the Justification of the Definition of Problems of Practice (N=53) Note. Problems of practice were often framed using multiple sources of evidence, so percentages of the sources of evidence for each category will add to greater than 100%. DISCUSSION Results indicated that the sampled dissertations in practice primarily relied on literature to provide evidence to frame PoPs and less frequently cited primary or secondary data as evidence. Due to the varied nature of PoPs and the contexts in which they are situated, the primary reliance on literature to substantiate PoPs is problematic. Whether theoretical or empirical in nature, the literature offers references that could provide a rationale or evidence of PoPs in a general sense, which is needed, but typically not to the particular context in which one selects a PoP, which is also needed. And although secondary data could be considered, primary data from the local context would seem to offer the most representative evidence of the PoP—a certain fit having been designed and collected by the investigator for the primary purpose of framing the PoP. With multifactor problems of practice, it may be best to consult multiple modes of evidence (Archbald, 2014) that include not only anecdotal data and literature, but also primary data that are directly relevant to the identified problem of practice. Evidence for Claims As scholars, we are prepared to offer evidence as justification for the claims that we make in our writing. Evidence can take many forms, but most often take the form of literature citations, theories, expert accounts, and/or empirical data that is either primary or secondary in nature. It is the presentation of the evidence by which the writer justifies his or her claims and the evaluation of such evidence by which the reader determines its validity (Booth et al., 2016). The credibility of the evidence can depend on various factors, with the circumstances under which the evidence was collected as key, and primary sources are generally considered more credible than secondary. And generally, if found to be valid, more evidence is better than less, particularly when it derives from a variety of sources. The fact that sources of evidence beyond literature are infrequently used to frame PoPs and systematic methodologies are used even more infrequently is concerning as it potentially limits the credibility of the PoPs. Improvements crafted for problems that are not fully understood stand the chance of ineffectiveness simply because they were developed to solve a problem that does not exist in the form that was originally conceived (Bryk et al., 2016). Implications and Recommendations for Practice Systematic methodologies are needed to frame PoPs to generate clear understandings of the root causes of problems to have hope of crafting effective and sustainable improvements. The exploration of a problem of practice would require collection of multiple forms of evidence that would be specific to that particular problem of practice. An inductive methodology that synthesizes the evidence would generate an understanding of the extent, scope, and underlying root causes of the problem of practice. Discussion of the specific methodologies that could be used to complete this goal is beyond the scope of this paper, but further exploration is needed to guide investigators in their methodological efforts to solve important educational problems. Leach, et al. Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 6, No. 4 (2021) DOI 10.5195/ie.2021.144 6 Within the systematic methodology of framing PoPs, clear and agreed-upon definitions of phrases like “local context”, “embedded in the work of the practitioner”, “persistent problem”, etc., are necessary to guide scholar-practitioners and their faculty mentors in the dissertation process. Although the nature of PoPs can vary based on topics and institutional program objectives, a common understanding facilitated by clear definitions could provide the standard needed to frame PoPs. And guidance regarding the evidence needed to substantiate these definitions could assist students and their faculty mentors in producing dissertations in practice that address PoPs that represent complex problems in need of improvement. As evidence takes a “backseat” to emotion and personal beliefs in the current post-truth era, methodologically sound systems to frame PoPs are arguably needed now more than ever to drive contextualized educational improvements that can be impactful, long-lasting, and sustainable. REFERENCES *References marked with an asterisk indicate studies included in the meta-analysis. *Alencastro, M. (2015). E Ho ‘oulu ‘la nᾱ kumu mauli ola hawai’i: Preparing Hawaiian cultural identity teachers. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/makalapua_ alencastre-alenc.pdf *Altman, M. R., & Pizano, A. C. (2016). Quality in early care and education: Multiple perspectives and critical considerations for professional development in a diverse context. Retrieved from https://search-proquest-com.zeus.tarleton.edu/docview/1847567976/ A6F3783D431F442APQ/1?accountid=7078 Archbald, D. (2014). The GAPPSI method: Problem-solving, planning and communicating. NCPEA Publishing. Arendt, H. (1951). The origins of totalitarianism. New York: Houghton Mifflin Harcourt Publishing. *Azziz, R. (2017). The nature of bias and its impact on disparate outcomes. Retrieved from https://www.cpedinitiative.org/assets/resource- center/docs/azziz_full_dip.pdf *Baete, G., Burks, J., & Pollio, M. (2012). Bringing urban high school reform to scale: Rapidly moving dramatic numbers of student to proficient performance. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/louisville_ burks_baete_polli.pdf Belzer, A., & Ryan, S. (2013). Defining the problem of practice dissertation: Where’s the practice, what’s the problem?. Planning and Changing, 44(3/4), 195-207. *Benedict, C. M. (2014). Professional learning community: Increasing efficacy for student success. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/benedict.pdf *Boeding, B. (2017). Supporting the camouflaged in transition: Serving student veterans with disabilities. Retrieved from https://www.cpedinitiative.org/ assets/resource-center/docs/boeding_full_dip.pdf Booth, W. C., Colomb, G. G., Williams, J. M., Bizup, J., & Fitzgerald, W. (2016). The craft of research (4th ed.). Chicago, IL: The University of Chicago Press. Bowen, G. A. (2009). Document analysis as a qualitative research methods. Qualitative Research Journal, 9(2), 27-40. *Brown, R. (2013). Equity for students of color through practitioner accountability. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/brown.pdf Bryk, A. S., Gomez, L. M., Grunow, A., & LeMahieu, P. G. (2015). Learning to improve: How America’s schools can get better at getting better. Cambridge, MA: Harvard Education Press. *Burke, A. (2012). From policy to practice: Implementing “move on when ready” at the local level in Arizona. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/burke_dip.pdf *Burt, T. D. (2014). From mission to measurement: Advancing postsecondary academic advising assessment and practice through evaluating program theory and outcomes. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/burt.pdf *Canelake, C. (2012). Implementing a standards-based teacher evaluation system: Learning experiences for administrators in an urban school district. Retrieved from https://www.cpedinitiative.org/assets/resource- center/docs/canelake_dip.pdf Carnegie Project on the Education Doctorate. (n.d.-a) About CPED. Retrieved from https://www.cpedinitiative.org/page/AboutUs Carnegie Project on the Education Doctorate. (n.d.-b). Database of Dissertations in Practice from CPED-Influenced EdD Programs. Retrieved from http://cpedinitiative.org/resource-library/dissertations Carnegie Project on the Education Doctorate. (n.d.-c). CPED framework. Retrieved from https://cdn.ymaws.com/www.cpedinitiative.org/resource/ resmgr/images/general_images/cped_infographic_5.png *Cato, M. (2013). Nursing students anxiety in simulation settings: A mixed methods study. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/cato.pdf *Chalkley, J., Lyles, T., & Stacy, J. (2012). Statewide communities of practice for excellence: A statewide program evaluation. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/ vcu_stacy.chalkley.lyles.pdf *Cicchino, M. (2014). Using game-based learning to foster critical thinking in student discourse. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/cicchino.pdf *Claravall, E. B. (2014). Creating and sustaining classroom knowledge through teacher research: Morphology as design intervention for struggling learners. Retrieved from https://www.cpedinitiative.org/assets/resource- center/docs/claravall.pdf *Conlon, S., Gallagher, T., & Hooper, S. (2012). The unrealized constructs of school reform. Retrieved from https://www.cpedinitiative.org/assets/ resource-center/docs/louisville_gallagher.conlon.pdf *Cooper, D. W. (2014). Exploring embedded remediation for community college career technical education pathways: Promising practices. Retrieved from https://www.cpedinitiative.org/assets/resource- center/docs/cooper.pdf Copland, M. A. (2000). Problem-based learning and prospective principals’ problem-framing ability. Educational Administration Quarterly, 36(4), 585- 607. *Creasia, C. R. (2014). The dynamics of instructional leadership & organizational structure in high performing urban schools. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/creasia.pdf *Diaz, C. S., Strang, M. L., Unger, A. L., & Van’t Hof, S. G. (2017). Utilizing the design thinking process to aid educators in their response to childhood lead exposure. Retrieved from https://www.cpedinitiative.org/assets/ resource-center/docs/diaz_strang_unger_and_vant.pdf *Eckerson, J. (2016). Teacher perspective on professional development needs for better serving Nebraska’s Spanish heritage language learners, Retrieved from https://www.cpedinitiative.org/assets/resource-center/ docs/eckerson_dissertation.pdf Gillham, J. C., Williams, N. V., Rife, G., & Parker, K. K. (2019). Problems of practice: A document analysis of education doctorate dissertations. Impacting Education: Journal on Transforming Professional Practice, 4(1), 1-9. *Gonzalez, A. (2014). Using funds of knowledge to build trust between a teacher and parents of language-delayed preschoolers. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/quintero_ gonzalez.pdf *Guan, Y. (2014). The effects of explicit listening strategy instruction on the listening comprehension of English as a second language (ESL) community college students. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/guan.pdf *Hall, R. A. (2014). Challenges on the road to school success for low socioeconomic and minority student: Dropout prevention and parent involvement. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/hall.pdf *Hanchi, N. M. (2013). A study of the implementation of the Ed.D. in educational leadership at the University of Central Florida 2010-2013: A profession. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/marsh.pdf *Hendron, J., Tolliver, S., Kim, A., & Deloatch, D. (2014). Ubiquitous computing in schools: A multi-case study of 1:1 districts. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/makalapua_%20alencastre-alenc.pdf https://www.cpedinitiative.org/assets/resource-center/docs/makalapua_%20alencastre-alenc.pdf https://search-proquest-com.zeus.tarleton.edu/docview/1847567976/%20A6F3783D431F442APQ/1?accountid=7078 https://search-proquest-com.zeus.tarleton.edu/docview/1847567976/%20A6F3783D431F442APQ/1?accountid=7078 https://www.cpedinitiative.org/assets/resource-center/docs/azziz_full_dip.pdf https://www.cpedinitiative.org/assets/resource-center/docs/azziz_full_dip.pdf https://www.cpedinitiative.org/assets/resource-center/docs/louisville_%20burks_baete_polli.pdf https://www.cpedinitiative.org/assets/resource-center/docs/louisville_%20burks_baete_polli.pdf https://www.cpedinitiative.org/assets/resource-center/docs/benedict.pdf https://www.cpedinitiative.org/%20assets/resource-center/docs/boeding_full_dip.pdf https://www.cpedinitiative.org/%20assets/resource-center/docs/boeding_full_dip.pdf https://www.cpedinitiative.org/assets/resource-center/docs/brown.pdf https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/assets/resource-center/docs/burt.pdf https://www.cpedinitiative.org/assets/resource-center/docs/canelake_dip.pdf https://www.cpedinitiative.org/assets/resource-center/docs/canelake_dip.pdf https://www.cpedinitiative.org/page/AboutUs http://cpedinitiative.org/resource-library/dissertations https://cdn.ymaws.com/www.cpedinitiative.org/resource/%20resmgr/images/general_images/cped_infographic_5.png https://cdn.ymaws.com/www.cpedinitiative.org/resource/%20resmgr/images/general_images/cped_infographic_5.png https://www.cpedinitiative.org/assets/resource-center/docs/cato.pdf https://www.cpedinitiative.org/assets/resource-center/docs/%20vcu_stacy.chalkley.lyles.pdf https://www.cpedinitiative.org/assets/resource-center/docs/%20vcu_stacy.chalkley.lyles.pdf https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/assets/%20resource-center/docs/louisville_gallagher.conlon.pdf https://www.cpedinitiative.org/assets/%20resource-center/docs/louisville_gallagher.conlon.pdf https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/assets/resource-center/docs/cooper.pdf https://www.cpedinitiative.org/assets/resource-center/docs/cooper.pdf https://www.cpedinitiative.org/assets/resource-center/docs/creasia.pdf https://www.cpedinitiative.org/assets/%20resource-center/docs/diaz_strang_unger_and_vant.pdf https://www.cpedinitiative.org/assets/%20resource-center/docs/diaz_strang_unger_and_vant.pdf https://www.cpedinitiative.org/assets/resource-center/%20docs/eckerson_dissertation.pdf https://www.cpedinitiative.org/assets/resource-center/%20docs/eckerson_dissertation.pdf https://www.cpedinitiative.org/assets/resource-center/docs/quintero_ https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase Using Evidence to Frame Problems of Practice Impacting Education: Journal on Transforming Professional Practice impactinged.pitt.edu Vol. 6, No. 4 (2021) DOI 10.5195/ie.2021.144 7 https://www.cpedinitiative.org/assets/resource-center/docs/hendron_et_ al.pdf *Henriquez, A. (2013). “I hope to be a symbol of encouragement”: Using craft in community colleges to facilitate student voice in the remedial studies debate. Retrieved from https://www.cpedinitiative.org/page/DiPDatabase [link changed after data collection and was not retrievable]. *Jaggers, A., & Royster, P. (2012). Putting time on your side: A system for getting all students on a college ready trajectory. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/louisville_ royster.jaggers.pdf *Jakubik, T. L. (2014). Graphic novels and the common core. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/jakubik.pdf Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational Technology Research and Development, 48(4), 63-85. *Kirchner, J. (2014). Student experiences of the community college developmental writing classroom. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/kirchner.pdf *Kline, N. C. (2014). The nature and degree of the relationship between chronic absenteeism and student achievement in a large metropolitan school district in Florida. Retrieved from https://search-proquest-com.zeus.tarleton.edu/docview/1860239472/ F52238B2E5F247D8PQ/1?accountid=7078 *Kobett, B. M. (2016). A university-led beginning teacher mathematics community. Retrieved from https://www.cpedinitiative.org/assets/ resource-center/docs/kobett_dissertation.pdf *Larson, J. L. (2013). Secondary mathematics teachers’ literacy professional learning: An amalgamation of adolescent literacy, mathematics teaching and adult learning. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/larson.pdf Lewis, C. (2015). What is improvement science? Do we need it in education? Educational Researcher, 44(1), 54-61. Retrieved from https://www.emmausseries2.org/uploads/2/0/5/2/20520752/lewis_2015.p df *Louis, T. (2014). Design and implementation plan for the “I am potential” financial literacy education program. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/louis.pdf Ma, V. W., Dana, N. F., Adams, A., & Kennedy, B. L. (2018). Understanding the problem of practice: An analysis of professional practice EdD dissertations. Impacting Education: Journal on Transforming Professional Practice, 3, 13-22. *Mataczynski, L. (2013). Advising and acculturation variables as predictors or sanctification, sense of belongings, and persistence among international undergraduates. Retrieved from https://www.cpedinitiative.org/assets/ resource-center/docs/mataczynski.pdf *Maltby, B., Richardson, D., Koontz, J., & Bhagat, R. (2014). A regional approach to school diversity: The possibility, feasibility, and desirability. Retrieved from https://www.cpedinitiative.org/assets/resource-center/ docs/maltby_et_al.pdf Mintrop, R. (2016). Design-based school improvement: A practical guide for education leaders. Cambridge, MA: Harvard Education Press. *Morrison, C. (2015). Co-constructing college-going capital in a rural high school English class. Retrieved from https://www.cpedinitiative.org/ assets/resource-center/docs/carissa_morrison-morrison.pdf *Mostofo, J. (2013). Using lesson study with preservice secondary mathematics teachers: Effects on instruction, planning and efficacy to teach mathematics. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/mostofo.pdf *Muser, H. (2017). A parent’s dream come true: A study of adult students who are parents and their academic engagement in high education. Retrieved from https://www.cpedinitiative.org/assets/resource-center/ docs/muser_full_dip.pdf *Olsen, B. D. (2016). Preservice and practicing teacher science inquiry projects: An analysis of their understanding of the inquiry process. Retrieved from https://www.cpedinitiative.org/assets/resource- center/docs/olsen.benjamin_fulldip.pdf Oxford Languages. (n.d.) Retrieved from https://en.oxforddictionaries.com/word-of-the-year/word-of-the-year-2016 *Palmer, R. (2014). Exploring online community among rural medical education students: A case study. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/cped_ award_presentation.pdf *Paredes, M. C. (2012). Parent involvement as an instructional strategy: Academic parent-teacher teams. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/paredes.pdf *Patel, A. (2014). An evaluation study of Ravenswood’s youth transitioning to and through high school. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/patel.pdf Perry, J. A. (2013). Carnegie project on the education doctorate: The education doctorate-degree for our time. Planning and Changing, 44(3/4), 113-126. Retrieved from https://www.researchgate.net/publication/272148530_The_Education_D octorate-_A_degree_for_our_times Perry, J. A., & Imig, D. G. (2016). What do we mean by impact? Impacting Education, 1. Retrieved from https://impactinged.pitt.edu/ojs/index.php/ImpactingEd/article/view/26/12 *Philadelphia, M. (2013). Will schoolbased online faculty development be an effective tool for their professional growth. Retrieved from https://www. cpedinitiative.org/assets/resource-center/docs/philadelphia.pdf *Rivera, N. (2013). Cooperative learning in a community college setting: Developmental coursework in mathematics. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/rivera.pdf *Riva, S. (2013). Deconstructing persistence in academic language among second-generation Latinos: How do second-generation Latino language minority community students alter their academic trajectories. Retrieved from https://www.cpedinitiative.org/assets/resource- center/docs/de_la_riva.pdf Rohanna, K. (2017). Breaking the “adopt, attack, abandon” cycle: A case for improvement science in K-12 education. In C.A. Christie, M. Inkelas, & S. Lemire (Eds.), Improvement Science in Evaluation: Methods and Uses. New Directions for Evaluation, 153, 65-77. https://doi.org/10.1002/ev.20233 *Rose, D. (2013). Enrollment decisions of the underrepresented: Recruiting and college choice at military institutions. Retrieved from https://www.cpedinitiative.org/assets/resource- center/docs/rose.pdf *Ruggles, T. (2012). Masters level graduate student writing groups: Exploring academic identity. Retrieved from https://search-proquest-com.zeus.tarleton.edu/docview/1010624372/ 2865932CF61B44D4PQ/1?accountid=7078 Schön, D. (1983). The reflective practitioner. New York: Basic Books. Schön, D. A., & Rein, M. (1995). Frame reflection: Toward the resolution of intractable policy controversies. New York: Basic Books. *Schwarting, J. (2014). Influencing social capital in times of change: A three pronged approach to instructional coaching at the middle school level. Retrieved from https://www.cpedinitiative.org/assets/resource- center/docs/schwarting.pdf *Spilde, A. (2013). Mathematics in a second grade classroom: The effects of cognitively guided problem solving. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/spilde.pdf Storey, V. A., Caskey, M. M., Hesbol, K. A., Marshall, J. E., & Maughan, B. D. (2015). Examining EdD dissertations in practice: The Carnegie Project on the Education Doctorate. International HETL Review, 5(2). Timperley, H. S., & Robinson, V. M. J. (1998). Collegiality in schools: Its nature and implications for problem solving. Educational Administration Quarterly, 34, 608-629. *Veith, V. (2013). Algebra for all and its relationship to English language learner’s opportunity-to-learn. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/veith.pdf *Walker, A. (2017). Transformative school leadership: A qualitative examination of school leaders’ sensemaking of lived racial experiences and racial inequities in education. Retrieved from https://cdn.ymaws.com/ www.cpedinitiative.org/resource/resmgr/docs/DiPs/Walker_Full_DiP.pdf *Walker, J. (2016). Brain-targeted early childhood beginnings: A case study in India brain-targeted early childhood beginnings: a case study in India. Retrieved from https://www.cpedinitiative.org/assets/resource- center/docs/cped_jimmie_walker_dissertat.pdf *Wallace, W. (2013). California accountability policy and intensification: A quantitative study of teachers’ perceptions. Retrieved from https://search-proquest-com.zeus.tarleton.edu/docview/1015179413/ DAE44916583E49E2PQ/1?accountid=7078 *Yavuz, O. (2014). Improving college readiness, pursuit, access, and persistence of disadvantaged students. Retrieved from https://www.cpedinitiative.org/assets/resource-center/docs/yavuz.pdf https://www.cpedinitiative.org/assets/resource-center/docs/hendron_et_ https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/assets/resource-center/docs/louisville_ https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/assets/resource-center/docs/jakubik.pdf https://www.cpedinitiative.org/assets/resource-center/docs/kirchner.pdf https://search-proquest-com.zeus.tarleton.edu/docview/1860239472/ https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/assets/%20resource-center/docs/kobett_dissertation.pdf https://www.cpedinitiative.org/assets/%20resource-center/docs/kobett_dissertation.pdf https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.emmausseries2.org/uploads/2/0/5/2/20520752/lewis_2015.pdf https://www.emmausseries2.org/uploads/2/0/5/2/20520752/lewis_2015.pdf https://www.cpedinitiative.org/assets/resource-center/docs/louis.pdf https://www.cpedinitiative.org/assets/ https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/assets/resource-center/%20docs/maltby_et_al.pdf https://www.cpedinitiative.org/assets/resource-center/%20docs/maltby_et_al.pdf https://www.cpedinitiative/ https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/assets/resource-center/%20docs/muser_full_dip.pdf https://www.cpedinitiative.org/assets/resource-center/%20docs/muser_full_dip.pdf https://www.cpedinitiative.org/assets/resource-center/docs/olsen.benjamin_fulldip.pdf https://www.cpedinitiative.org/assets/resource-center/docs/olsen.benjamin_fulldip.pdf https://en.oxforddictionaries.com/word-of-the-year/word-of-the-year-2016 https://www.cpedinitiative.org/assets/resource-center/docs/cped_award_presentation.pdf https://www.cpedinitiative.org/assets/resource-center/docs/cped_award_presentation.pdf https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/assets/resource-center/docs/patel.pdf https://impactinged.pitt.edu/ojs/index.php/ImpactingEd/article/view/26/12 https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/assets/resource-center/docs/de_la_riva.pdf https://www.cpedinitiative.org/assets/resource-center/docs/de_la_riva.pdf https://doi.org/10.1002/ev.20233 https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://search-proquest-com.zeus.tarleton.edu/docview/1010624372/%202865932CF61B44D4PQ/1?accountid=7078 https://search-proquest-com.zeus.tarleton.edu/docview/1010624372/%202865932CF61B44D4PQ/1?accountid=7078 https://www.cpedinitiative.org/assets/resource-center/docs/schwarting.pdf https://www.cpedinitiative.org/assets/resource-center/docs/schwarting.pdf https://www.cpedinitiative.org/assets/resource-center/docs/spilde.pdf https://www.cpedinitiative.org/assets/resource-center/docs/veith.pdf https://cdn.ymaws.com/%20www.cpedinitiative.org/resource/resmgr/docs/DiPs/Walker_Full_DiP.pdf https://cdn.ymaws.com/%20www.cpedinitiative.org/resource/resmgr/docs/DiPs/Walker_Full_DiP.pdf https://www.cpedinitiative.org/assets/resource-center/docs/cped_jimmie_walker_dissertat.pdf https://www.cpedinitiative.org/assets/resource-center/docs/cped_jimmie_walker_dissertat.pdf https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://search-proquest-com.zeus.tarleton.edu/docview/1015179413/%20DAE44916583E49E2PQ/1?accountid=7078 https://search-proquest-com.zeus.tarleton.edu/docview/1015179413/%20DAE44916583E49E2PQ/1?accountid=7078 https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase https://www.cpedinitiative.org/page/DiPDatabase Lesley F. Leach Tarleton State University Credence Baker Tarleton State University (1) Catherine G. Leamons Tarleton State University (2) Phillis Bunch Tarleton State University (3) Jesse Brock Tarleton State University (4) Carnegie Project on the Education Doctorate Dissertation in Practice Problems of Practice Need for Evidence in Post-Truth Era Study Purpose method Results Discussion Evidence for Claims Implications and Recommendations for Practice References