Focusing on Actors in Context-Specific, Data-Informed Theories of Change 20 _______________________________________________________________________________________________________________________ Global Education Review is a publication of The School of Education at Mercy College, New York. This is an Open Access article distributed under the terms of the Creative Commons Attribution-Noncommercial 3.0 Unported License, permitting all non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. Citation: Smith, Mariam, and Ball, Jessica (2020). Focusing on Actors in Context-Specific, Data-Informed Theories of Change to Increase Inclusion in Basic Education Reforms. Global Education Review, 7 (1). 20-40. Focusing on Actors in Context-Specific, Data-Informed Theories of Change to Increase Inclusion in Basic Education Reforms Mariam Smith Learning Loop Consultancy, Sweden Jessica Ball University of Victoria, Canada Abstract This article describes an actor-focused approach to creating a theory of change that is context-specific, grounded in an understanding of local historical, cultural, linguistic, and social realities, and inspired by a commitment to social justice in education reform. In 2019, the authors completed a research evaluation of the Government of Cambodia’s inaugural Multilingual Education National Action Plan designed to increase inclusion of Indigenous children in basic education. The project mandate included constructing a theory of change. Using the participatory approach of Outcome Harvesting, data were obtained about behavioral changes among actors implementing and affected by the plan. Qualitative data analyses identified 115 behaviors distributed across 15 categories of actors in the education system, and uncovered assumptions and experiences of change processes and relationships. The authors created a generic theory of change mandated by the commissioning body for the evaluation. It was unidirectional and institution- centered, focused on objectives, strategies, outputs, and outcomes. To provide a more nuanced, inclusive, context-specific and potentially useful representation of change processes, the authors drew upon the behavior change data to construct a second, actor-focused theory of change. Additionally, the authors constructed a third theory of change showing education strategies in the current context compared to internationally accepted best practice in multilingual education. This study illustrates a focus on manifest behaviors and relationships among actors in theories of change. Actor-focused frameworks that describe situationally specific, participatory action and reciprocal learning can promote inclusive, sustainable, systems-level change toward children’s right to meaningful quality basic education. Keywords Theory of Change; Indigenous; Actor-focused; Complexity; Outcomes; Situated Learning; Multilingual education; Systems; Participation; Social justice; Rights; Social change Introduction This article describes Theories of Change (ToCs) as tools which can be more or less useful for change agents depending upon how the ToC is used and its design features. Six features derived from literature are described which can encourage or limit a ToC’s potential use for accountability, learning, and engagement toward social justice in education reform. The article then sets the scene for a case example by describing the context of Indigenous education and rights. The case describes our experiences of creating ToCs for an Indigenous language-in- education initiative in Cambodia. The agency- mandated task of constructing a ToC evolved into the creation of three types of ToCs: (1) a generic, institution-focused ToC; (2) an actor- focused ToC; and (3) an analysis of strategies against best practice in multilingual education. The article then discusses the potential benefits and limitations of various approaches to ToC, particularly the actor-focused ToC, building further on the six features introduced in the beginning of the article by drawing from literature and our own experiences during the case example. The article concludes that, while all ToC approaches are limited in representing Focusing on Actors in Context-Specific, Data-Informed Theories of Change 21 realities, an actor-focused ToC is a potential tool for changing power dynamics and inclusion of actors in education reform. Introducing the ToC as a Tool Theories of Change (ToCs) can provide programs with clarity on how change is believed to transpire. In recent years, ToCs have become a significant tool for communication (Valters, 2015). ToCs are found in program documents such as project proposals, strategic plans, and evaluation reports. Typically, those involved in program design formulate the logic of the program, outlining the context, problems, assumptions, activities, expected outputs, risks, outcomes, and goals. A ToC often contains agreed upon terms and statements defining the presumed or intended pathway from the assumptions and problems to the desired vision or goal. These are often further developed for a monitoring framework. Most often, these center on measurable, projected outputs and they work as an accountability system between implementing agencies and funding agencies. Typically, there is pressure from each part of a funding hierarchy to keep implementers accountable to detailed program activities, with an assumption that these “deliverables” will result in targeted outcomes, such as an increase in the number of children with access to preschool or to multilingual education (MLE) in primary school. A ToC can support the accountability system and can be used as a tool for engagement and learning. At the same time, we have seen how ToCs can limit engagement and learning, and also preempt emergent, productive processes of intended and unintended interactions among stakeholders and behavior change. Whether a ToC is more or less useful for engagement, learning, and accountability depends on a number of features derived from literature and our own experiences in education reforms. These include, but are not limited to, (a) the form of the ToC, whether generic or fitted to a specific context and purpose, whether written or visual or both, and what languages it is presented in for diverse stakeholders’ engagement; (b) whether the necessary simplification of the change process shown in a ToC is conceived as a definitive encapsulation of a finite process or, instead, as an abbreviated and mutable notation for ongoing deliberation about a dynamic and complex process; (c) whether the ToC, from its inception, is conceived as a tool for generating engagement of key actors in an authentic, collaborative process of generating change; (d) whether the creation of a ToC represents a fixed and linear sequence culminating in summative valuation against targeted outcomes, just as the earlier Logic Models were intended, or instead represents an iterative, networked, and circular process; (e) whether the ToC is entirely context-based or borrows in part or wholly from external contexts and models that are assumed to generalize to the context at hand; and (f) whether the ToC is understood as a heuristic tool or as a positivist depiction of reality. For each of these features, investigators face dilemmas and choices to ensure that the ToC stimulates communication, collaboration, learning, and accountability among actors. We contend that if these issues are deliberated with considerations of the specific purpose and context where ToCs are meant to be useful, ToCs can contribute to justice and inclusion in social change processes, such as Indigenous participation in education reform. In the following discussion, we elaborate upon each of these six features and then illustrate them with reference to our recent evaluation of a government intervention to increase Indigenous participation in basic education in Cambodia. 22 Global Education Review 7 (1) The Form of the ToC A skilled narrator can express logic in writing, communicating the theory of how change will happen or has happened. Although ToCs have long been expressed in narrative form, diagrammatic representation of ToCs have become popular (Davies, 2018). With limited space in diagrammatic forms, choices of words and symbols communicate elements, interactions, and outcomes that are viewed as essential or most important. Generally, only the most central actors are visible. This can limit the sense of a ToC as an invitation for engagement and strategic action, since those who are not identified in the diagrammatic change process may feel devalued, and may in fact be overlooked in decision-making forums and program activities as the ToC is used as a reference point for implementation and accountability. In visual form, ToCs can easily be made accessible as a shared focus for dialogue and reflection. ToCs are, however, cultural artifacts. Their form embodies the culture of one or several actors (and often the institutions they represent). This enculturation will influence the effectiveness of the ToC as a communication tool within and across cultures, institutions, and communities. For example, in many Indigenous cultures, change is conceived as cyclical and interdependent and often represented in the form of concentric circles (Hill & Stairs, 2002; McGregor, 2004). Few examples of ToCs take the form of circles or spirals, or highlight the interdependence of human and non-human actors that are seen in some cultures as mutually influential in determining outcomes. The ToC as a Tool for On-Going Deliberation to Work with Complexity Development processes that call upon multiple actors to change their behavior are, by definition nonlinear and unpredictable, and therefore complex (Carden & Earl, 2007; Conlin & Stirrat, 2008; Smith, 2017). In complex social change, there are few solutions that are assured a priori because there are always multiple belief systems and perspectives, and people make both rational and irrational decisions (Ramalingam, 2013; Thaler, 2009). Besides perspectives, there are unique factors in each context adding to its complexity. It is generally understood that complexity means that change involves multiple actors and perspectives, and it is dynamic; each new development in a complex change process yields new insights, risks, objectives, opportunities, and power relationships, all calling for negotiation. As we have experienced, practitioners working within complex social change often know all too well the unique, interrelated, emergent, and dynamic processes in their particular program context. They are likely to respond negatively to the development and use of ToCs in which inevitably simplified representations of change processes seem uninformed and unsympathetic to the unpredictability of many elements and interactions within the context or process of change. Demands for compliance to a prescribed and ossified ToC may be received as a threat to adaptive management and effective practices involving multi-stakeholder partnerships. Complex social change requires robust engagement in learning such that patterns of change can be explored through an ongoing process of small, iterative cycles of action and reflection. Institutional program logic, however, tends to rely on assumptions conceived outside the immediate context and invoke the (suspect) concept of best practices or aspirations prized among development agencies rather than probing ideas and innovation, which is required in complex social change processes. Focusing on Actors in Context-Specific, Data-Informed Theories of Change 23 The Collaborative Purpose of the ToC In an increasingly connected world, actors interact and social problems intersect in various ways, requiring collaboration. As Thaler (2009) emphasizes in his groundbreaking economic models, humans are the most important agents of change, and our models have to centralize how we make decisions (often irrationally) and act to create particular kinds of (intended and unintended) outcomes. We contend that the main goal of a ToC is to stimulate communication, collaboration, learning, and accountability among actors. A ToC may be produced by a program design team or consultant with or without the participation of those implementing the program or affected by the program. A ToC may be conceived through collaboration, and may be used as a stimulus for reflection, learning, and further collaboration. When all parts of an envisioned change process are exposed in a clear ToC, stakeholders can see their roles (or lack thereof) in the change process and begin dialogue with a shared understanding of what the program is supposed to do and achieve. Choices about providing opportunities for collaboration will affect the legitimation and use of a ToC by various stakeholders for learning and accountability. We support the view of Patton (2008), who advocates utilization- focused evaluation, in which the primary intended users of evaluation are clearly identified and personally engaged at the inception of the evaluation process to ensure that their needs and goals are identified. In our experiences in education reform, stakeholders whose behaviors will enable or detract from the success of a planned education reform are more likely to engage and feel ownership over the change process if they have been actively involved in conceptualizing the ToC for the intervention. The Fixed and Linear or Iterative and Circular Change Process ToCs are embedded in world views and organizational cultures. Vertical and hierarchical cultures may demand an institutionally- generated ToC during the planning process, followed by periodic reporting on progress toward predetermined outcomes in a linear, logic model. However, ToCs can be useful to forecast and reflect upon change processes at any point in a program (Davies, 2018). Organizations recognizing complexity and uncertainty in change processes may actively promote a generative process for the creation and use of ToCs. Multiple perspectives, directions, and feedback loops can iteratively reveal new patterns to support learning (Kurtz & Snowden, 2003). We argue that this kind of process-oriented use of a ToC as a tool can guide ongoing adjustments to program interventions and can be useful to accountability systems, including monitoring and evaluation. The ToC as Context-Based or Borrowing From Other Contexts Each context has its own history, set of values and perspectives, and dynamic relationships which affect strategies and outcomes. Funnell and Rogers (2011) describe multiple ways of depicting a ToC depending on the context, type of program, needs, goals, and complexity of the intervention and the context (Rogers, 2014). ToCs can be created by external experts or by actors from within a local context. External input can inspire and inform so-called best program models or practices visible in a ToC. However, practices that are “best” in one context do not necessarily work across diverse contexts. Context-specific knowledge and participation is essential for the validity and usefulness of the ToC. 24 Global Education Review 7 (1) The ToC as a Heuristic Tool or a Positivist Reality A ToC produced on the basis of substantial, relevant, and current data from a local context can be conceived as a positivist reality or, from a post-positivist stance, as a heuristic tool. From our post-positivist and systems perspective, a ToC (even when informed by data) will only be a representation of particular perspectives of a reality; it is a constructed system, and not reality itself (Williams & Hummelbrunner, 2010). When a ToC is taken as an accurate map of reality, this can put pressure on design and performance of program activities, as actors attempt to comply with what has been presumed, prescribed, predetermined, and predicated by the ToC. On the other hand, conceptualizing a ToC as a heuristic tool for ongoing engagement and learning, as well as for accountability, gives room for diverse perspectives and ongoing modifications to roles and goals as new conditions, actors, potentially effective actions, intended and unintended outcomes, and other factors come to light. A helpful metaphor may be a comparison of jazz with classical music, where a ToC for educational development work allows for improvisation, like playing jazz, rather than being constrained by a predetermined musical score. The remainder of this article illustrates these considerations with reference to our recent experience evaluating a policy intervention to promote the language rights of young Indigenous children in Cambodia. This project provides a vivid example of complex social change processes. Based on this and other experiences in education for social justice and sustainable development, we argue that ToCs can be useful tools in contexts where complexity is recognized in rights-based program planning, evaluation, research, and development, but only if the following conditions are present: (a) timely and authentic opportunities for participation by local actors; (b) acknowledgement of the ecosystems of various actors and the knowledge and belief systems of those whom the intervention is meant to serve; and (c) an understanding that a ToC is an iterative, heuristic tool for learning, collaboration, and accountability. Indigenous Education and Rights In 2018, we were commissioned to evaluate Cambodia’s Multilingual Education National Action Plan (MENAP) 2015-2018 and the implementation of this plan for inclusive quality education for Indigenous children. This education reform in Cambodia was seen as a complex social change process for several reasons, including: (1) the normative nature of education; (2) the contested nature of Indigenous inclusion/exclusion in education decision-making; and (3) the institution-driven nature of most development assistance systems. Educational Relevance for Indigenous Contexts Education is a normative function in society and is deeply nested in cultural and societal values and aspirations. Through the education system, normative values are transmitted, defining what education is meant to do. Education can be for economical, technical, or other social goals (Dahlstedt & Olson, 2013). It can be taken as axiomatic that an education system will not work the same way in different contexts. Most ethnic and cultural groups have unique worldviews, languages, approaches to teaching and learning, and goals for formal and informal education. Also, the education program needs to respond to the historical and contemporary relationships and place. With this in mind, it might be expected that the normative values of a mainstream education system may not match the values and goals of Indigenous Focusing on Actors in Context-Specific, Data-Informed Theories of Change 25 Peoples who are the intended beneficiaries of the education system or reform (Battiste, 2013). In education for Indigenous children, programs need to account for Indigenous forms of knowledge, participation and leadership (Hill & Stairs, 2002; McGregor, 2004). An Indigenous child needs an education fit for the realities and ways of living of their culture within their particular context, which is dynamic and rapidly changing. While we recognize the great diversity of Indigenous Peoples globally, the shared experience of colonization has left many communities facing common challenges. In light of this, it is worthwhile to explore how ToCs that can be optimally utilized to advance the goals of particular communities who are facing similar struggles. Indigenous lifestyles are often strongly connected to biodiversity hotspots around the world, where Indigenous Peoples hold extensive, context-specific knowledge about the local environment. When development organizations aim to create a “better world” through “quality education,” Indigenous input is required to confirm their agreement with the ideological agenda and associated development targets and strategies (Ball, 2005; Smith, 2017). Through the delivery of education, teachers communicate a depiction of history, the present, and possible futures which will shape young people as citizens. Education decision-makers, curriculum writers, and teachers need to be held ethically and politically responsible for the legitimacy and utility of these depictions (Dahlstedt & Olson, 2013). Indigenous Inclusion/Exclusion in Decision-Making The contexts in which most Indigenous Peoples live is dominated by majority ethnic groups and their cultures. Indigenous perspectives may not be heard and Indigenous peoples’ inclusion in education decision-making forums is often only symbolic. In projects and policies affecting Indigenous Peoples, there are potentially implicit agendas of nationalism and assimilation, and financial motivations promoting or disrupting Indigenous participation (Wong & Benson, 2019). Added to this complexity, Indigenous Peoples have typically experienced significant environmental and cultural losses, further challenging trust, relationships, and the ability to work collaboratively with those wishing to implement education reform. Development Assistance Systems Globally, education for Indigenous children faces significant challenges (Heyman & Cassola, 2012). Programs to solve seemingly intractable problems of access, relevance, and achievement require far more funding and technical support than is allocated in most countries. Solutions also depend on political will to fulfill Indigenous parents’ and children’s rights to participate in education decision-making in order to ensure that education reforms meet their self-identified needs and goals (Battiste, 2017). Further, policies regarding the languages in which education is delivered are an exercise of power that often violates Indigenous rights (Cummins, 2000). The development assistance system itself has its organizational comings and goings, funding streams, and silos guided by dominant discourses and popular strategies, requiring political negotiations from which Indigenous Peoples are often far removed. These three factors, along with others too numerous to elaborate here, indicate a need for negotiation of power and identity in education initiatives. Rather than merely rolling out education services including new initiatives, there is a need to convene and support networks of stakeholders able to negotiate the normative role of education with and for Indigenous people (Ball, 2005). Planning tools that support 26 Global Education Review 7 (1) collaboratively created curricula, delivery, and accountability processes are needed (Ball, 2004). In efforts to implement potentially fruitful policy and practice reforms, we argue that a solely agency-driven, linear approach to ToC can be counterproductive. In newly independent countries such as Cambodia, this approach reinscribes exclusionary, colonial practice. As noted earlier, generic ToCs can alienate key actors as they perpetuate a patronizing, authoritarian stance that privileges the needs of funders and governments over citizen needs for self-determination, civic participation in government decision-making, and strengthened local capacity for engagement in education that is meaningful within specific community and cultural contexts. We argue that for ToCs to be useful in supporting complex social change that realizes Indigenous children’s rights to quality education, they must be inclusively legitimated, collaborative, context- specific, and process-oriented in ways that are accessible and meaningful to actors within the local system and particularly to Indigenous Peoples. In our evaluation of a rights-oriented education policy initiative in Cambodia, we were asked to retroactively construct a ToC for the purpose of understanding and validating strategies and activities undertaken over the previous five years to carry out the initiative. In the next section, we describe how we aimed to make the ToC tool useful by drawing upon a process of participatory data collection, and by centering the interdependency of actors and their unique roles in implementing the new policy. A Case Study of Creating a ToC for an Indigenous Language-in-Education Initiative Evaluation Context In Cambodia, the Ministry of Education, Youth and Sport in collaboration with UNICEF Cambodia and CARE, and in consultation with other stakeholders in Cambodia, launched a Multilingual Education National Action Plan, 2014-2018, (MENAP) (Royal Government of Cambodia, 2015). The goal was to increase the inclusion of Indigenous children in education by providing bilingual education involving Khmer (the dominant language) and five Indigenous languages (Bunong, Kavet, Kreung, Tampuan, and Brao) for children in preschool and lower primary education in five provinces. This plan was unprecedented in Southeast Asia for its government commitment to using ethnic minority languages to promote equity (Ball & Smith, 2019). We co-led an independent evaluation of this plan to provide evidence-based conclusions on the extent and quality of its implementation and to make recommendations for future action. We were assisted by an in- country team of nine Indigenous women and men who, combined, were proficient in the five languages used in MLE. Evaluation Process The evaluation used a mixed-methods, iterative-inductive approach. Priority was placed on Indigenous participation and methods that are becoming recognized as good practice in data gathering involving Indigenous people (Ball, 2005; Battiste, 2013). Data were collected in the provinces of Ratanakiri, Mondulkiri, Stung Treng, and Kratie where MLE had been introduced for at least one year. The aim was to assess results against objectives, review strategies and support, and document lessons learned about implementation. Seven hundred Focusing on Actors in Context-Specific, Data-Informed Theories of Change 27 people engaged in 84 focus group discussions, six Outcome Harvesting workshops, and key informant interviews. Surveys and direct observation were also used. Participants included Indigenous children and parents, MLE teachers, school principals, school support committees, Indigenous Peoples’ organizations, commune council members, village leaders, District Offices of Education, Provincial Offices of Education, staff of the Ministry of Education, Youth and Sport and UNICEF, and international, national, and local non- governmental organizations. The evaluation team visited 24 MLE primary schools in 14 provincial districts, 11 MLE preschools in eight districts, and four non-MLE primary schools for comparison. Secondary data included a review of over 100 in-country planning, policy and evaluation documents, international studies of MLE and language-in-education policies, costing data, and quantitative education management data, as available. The evaluation used a participatory, qualitative research approach, Outcome Harvesting (OH), as the main tool. OH is an approach inspired by Outcome Mapping (Earl, Carden, & Smutylo, 2001), which acknowledges that sustainable development depends on the behavioral change of multiple actors. Outcome Mapping is one type of actor-focused theory of change where aspirational outcomes are described in advance by those knowledgeable about a local context to help practitioners be aware of changes toward the desired vision. Exploring concrete observable changes in behavior, relationships, practices, and policies can support actors in understanding systemic progress toward a development goal. OH is a utilization-focused evaluation approach in which positive and negative outcomes are “harvested” through a facilitated and participatory process (Wilson-Grau 2018). A utilization-focus means that the approach is intended to make evaluation experiences and results useful for decision makers. Participation throughout an evaluation process is a key underlying aspect of a utilization focus, ensuring that program learning is embedded in the process rather than merely a shelved product of legitimization and accountability for donors (Patton, 2008). During field work in the provinces and through grassroots-level focus group discussions, the team listened to the perspectives and outcomes known to students, teachers, school leadership, parents, and other community members. Using a timeline drawn on large rolls of paper, participants used drawings, symbols, and words to map the most important changes in their communities. From these inputs, the team built up a portfolio of questions, themes, and outcomes from various stakeholders’ perspectives to bring to province- level workshops. Representatives from each village and provincial government authorities in education participated in the workshops to verify and substantiate outcomes, build consensus, and provide triangulation. The team also interviewed representatives of participating nongovernment and government organizations at provincial and national levels to understand activities that had been undertaken by various actors, interactions among actors and their activities, and potential contributions to emerging outcomes. Participants described bottlenecks, barriers, enabling factors, opportunities, and additional perspectives on the significance of various inputs, outputs, and outcomes. A final, multi-stakeholder “discovery” workshop was held in the national capital with participants from four provinces, including Indigenous parents and representatives of Indigenous Peoples’ Organizations. 28 Global Education Review 7 (1) Reconstructing ToCs for the Education Intervention A ToC was lacking in the initial plan for the MENAP. As noted, we were asked to construct a ToC retrospectively, using a generic, institutional model, in order to clarify the planned intervention and validate strategies and activities. Since we are convinced of the utility of more context-specific, data-informed, actor- focused approaches to ToCs, we also constructed two additional ToCs, drawing on findings from data gathered in the field. Presented subsequently, the three ToCs we produced included: (1) a generic ToC; (2) an actor-focused ToC; and (3) an analysis of strategies against best practice in MLE. The ToC constructed using the generic model required by the commissioners of the evaluation is shown in Figure 1 (see Appendix). A strength of the generic ToC, which is commonly required in evaluations, is that it provides a snapshot of the intentions and main strategies of the MENAP. However, a generic, unidirectional ToC such as the one shown in Figure 1 excludes actors and excludes their behaviors (activities) intended to realize intended outcomes, and relationships among actors and their activities. Our actor-focused ToC is shown in Figure 2 (see Appendix). This ToC strongly built on the findings of our data collection in the field. This approach shows key relationships among actors who were implementing intended strategies on the ground. We perceived that this representation could inform the development of a second five-year plan for MLE and support a theory of action. The actor-focused ToC has two parts. One part shows an overview of the actor-focused ToC, which recognizes the broad array of actors and their relationships to one another in order to implement MLE. In this form, the ToC can be seen as a stakeholder analysis or a network analysis in regard to a particular change, which is often a useful way of expressing a ToC (Davies, 2018). Our ToC drew upon findings about the various actors already included in the MENAP and their current relationships, those actors who were phasing out, as well as those not yet included but with potential significant contributions for future planning. For example, as shown by broken red lines in Figure 2, Indigenous Peoples’ Organizations (IPOs) were excluded from the MENAP, but during field work they came forward with a strong demand to be involved and with demonstrable evidence of unique knowledge of the contexts and languages implicated in the MENAP. The second part of the actor-focused theory of change, illustrated in Tables 1, 2, and 3, zoomed in on each type of actor, describing the specific, concrete, new behaviors and new relationships needed, inspired by Outcome Mapping approaches. For each actor in the ToC, the evaluation gathered evidence of emerging behaviors (yellow), observed behaviors (green), behaviors not yet seen but necessary for effective implementation of the MENAP (orange), and behaviors that contradict behaviors necessary for effective implementation of the MENAP (red). Table 1 shows an example of a set of intermediate outcomes developed for one of the 25 categories of actors in the system (in this example, teachers). The active verbs describe emerging and observed behavior among MLE primary school teachers. During the evaluation, we heard about current limitations and hopes for the future, gaining ideas for future progression. Combined with research and practice-based literature about MLE and the consultants’ own experiences, these findings were used to map future behavior changes on the parts of various actors in order to achieve the desired goal of quality basic education for Indigenous children in Cambodia. Focusing on Actors in Context-Specific, Data-Informed Theories of Change 29 Table 1: MLE Primary School Teachers’ Implementations Related to the MENAP Expected change (intermediate outcome) Current status Confidently using the mother tongue as the language of instruction. Observed Employing teaching techniques that use local cultural resources and expertise. Observed Creating appropriate teaching and learning materials in the mother tongue, collaborating with others. Emerging. Requesting more training in mother tongue literacy skills. Using the MLE curriculum. Emerging. Using CARE curriculum but in some schools adding MoEYS standard curriculum as well. Communicating with parents regularly. Emerging. Communicating with parents only about attendance. Developing close relationships with children. Observed Note. MoEYS=Ministry of Education Youth and Sports; CARE=Cooperative for Assistance and Relief Everywhere Table 2 shows another set of intermediate outcomes—in this example for MLE District of Education officers. For this actor category, intermediate outcomes were a blend of emerging, observed, and not yet seen behaviors. Table 2: District of Education Officers Expected change (intermediate outcome). Current status Investigating expansion of MLE schools and preschools. Observed Coordinating and liaising for MLE support and expertise with IP organizations and networks. Not seen Administratively supporting MLE schools. Observed Engaging communities to get involved in supporting MLE programs. Observed Providing valuable and culturally relevant support to teachers in MLE teaching methodology. Requesting more training in MLE. Core trainers requesting Mother-tongue literacy skills. Some only communicating value of Khmer literacy. Inspecting and testing children using tools appropriate for MLE Not seen Collecting data for MLE reports useful for learning about progress (identifying indigeneity). Emerging 30 Global Education Review 7 (1) There were other actors not yet recognized as change agents in the MENAP. Table 3 describes the current status of behaviors among Indigenous Peoples Organizations (IPOs) based on our research findings. These IPOs ranged from informal networks to more formal Indigenous associations and organizations. Our understanding was that the data showing an absence of behavior change on the parts of IPOs did not necessarily indicate an unwillingness to contribute or to change, nor an active resistance from institutional MENAP implementers. Rather, IPO’s lack of behavior change was a result of IPOs not being recognized in the original plan and not being explicitly recognized in any written or diagrammatic form in the government plan. Table 3: Indigenous People Organizations Expected change (intermediate outcome) Current status Engaging in decisions regarding Indigenous children’s education, including curriculum development at national level. Not seen Engaging in decisions regarding Indigenous children’s education at provincial and local levels. Not seen Mobilizing Indigenous communities in understanding and engaging in MLE. Not seen Representing the voices and insights of Indigenous groups regarding MLE effectiveness and relevance. Not seen Mentoring and reassuring teachers and School Support Committees that MLE is authorized by national government. Not seen Advocating for supportive policy and guidelines for implementation of MLE. Not seen Note. MoEYS=Ministry of Education Youth and Sports; CARE=Cooperative for Assistance and Relief Everywhere The actor-focused ToC shown in Table 3 shows the current status of intermediate (compared to final) outcomes and what we perceived as a need to include Indigenous Peoples’ perspectives on change, based on the research findings. Among the 25 categories of actors in our actor-focused ToC, some were not yet visible in the MENAP but were found to provide enabling factors or barriers. Examples of these include the Khmer population of children and Khmer guardians in Indigenous villages, universities, and provincial teacher training colleges (Khmer are the dominant ethnic and language population). The actor-focused ToC shown in Figure 2 and Tables 1, 2 and 3 illustrates how a data-driven, context-specific, actor-focused ToC can provide clarity on key actors or change agents, roles, inter-relations, progress, and expectations. Our ToC model documents how each actor is progressing, rather than wishful thinking (strategies weakly developed to address the needs), high-level impressions, claims about behavior change that cannot be verified through triangulated data gathering and consensus building, or unrealistic expectations. A third way of approaching the logic of the MENAP was to analyze the planned strategies and approaches in the plan with reference to internationally recognized MLE good practices defined by ten main components outlined by UNESCO (2018). Figure 3 (see Appendix) shows strategies (left column) explicit in the MENAP linked to a set of intermediate changes (right column) that the evaluation team deduced Focusing on Actors in Context-Specific, Data-Informed Theories of Change 31 from the strategies and other components of the MENAP. This ToC served as a critical assessment of the MENAP compared to established MLE best practices, making visible what strategies had been employed in the MENAP and which had not been employed. Three components of good practice were not included in the MENAP and are therefore not shown in the ToC presented in Figure 3: preliminary research, acceptable alphabets and supportive mother tongue-based MLE policy and legislation. The strength of this approach to a ToC is the connection to the accumulated practice-based evidence documented by other MLE actors in other places. Discussion In this section, we reflect upon some of the considerations discussed in our introduction with regard to whether a ToC is useful for learning, engagement, and accountability in education reforms. The Form of the ToC and Its Use in Complex Change First, we conclude from this and other experiences in education reform that a ToC can be useful for a community of practice with a common endeavor, serving as a form of reification (or artifact for those involved) making explicit the driving values, belief systems, and aspirations (Wenger, 1998). People are shaped by many cultural tools (Wertsch, 1998), visible and invisible. A ToC can be used as a visual exposé, revealing how a group believes change will happen. A data-based, context-specific ToC will also expose how various actors behave and the reciprocal causal impacts of their behaviors. The ToC can, however, communicate an unfounded conceptualization of change as a linear process, and the misperception that implementation activities correlate one-to-one with logical framework objectives (Davies, 2004). Davies (2005) contends that “removing the one-directional nature of change leads us from thinking about a chain of events to a network of events, and from a chain of actors to a network of actors” (p.134). In our example, there was an institutional perspective in Cambodia that because the MENAP was a national action plan, then the government was the main actor as a service provider. We and our Indigenous team colleagues, however, were acutely aware of the risks of not sufficiently seeing and making visible the many non- government actors whose behaviors could enable or constrain implementation of the planned change and goal attainment. The ToC needed to be congruent with our evaluation process, acknowledging complexity and diversity of experiences at different levels of the system and by different categories of actors, as Vogel (2012) suggests in her review of ToCs. In our actor-focused ToC, the simplification of behavioral outcomes was a network analysis, consistent with recommendations by Davies (2004, 2005), and informed by complexity science and development knowledge. Our ToC built on the data on behavior and relationships acquired during the Outcome Harvesting process. Further, ToCs, in any form, are limited in their ability to communicate intermediate outcomes and the complexity of connections. As Ramalingam (2013) cautions, they often present what is written in a logical framework, as a simple and discreet, carved off version of complex social reality. Relationships are often ignored in ToCs. Also, as Davies (2018) comments, visualizations of ToCs tend to emphasize the content of the composite boxes, while the meaning of an arrow (signifying a connection or relationship) remains unexplicated and, too often, unexplored. When there is little political space for Indigenous development and participation in 32 Global Education Review 7 (1) education, it may seem advantageous to choose consensus around ambiguous program logic. We argue, however, that making terms in a ToC concrete, in terms of operationally defined behaviors (or activities), can help to engage more actors with experience and deep knowledge of local contexts. Such clarity should support local actors to meaningfully engage and keep government actors accountable to commitments they make. Representation of the findings of our evaluation in the actor-focused ToC, for example, can inspire education officials to include a wider range of actors, including those who are intended beneficiaries of the education initiative, and to show actors and their inter-relationships in their planning tools in the future. Among other things, this has implications for distribution of funds, since actors whose behaviors are explicitly recognized as necessary for successful implementation of a planned change need monetary and other forms of institutional support for the full and meaningful contributions they can make. Davies (2004) urges us to adapt “our representational devices to the different environments in which they are being used, and not insist on one standard model” (p.103). There are advantages of simplifying a representation in order to encourage dialogue and meaning- making. The constraint, or risk, is that the representation becomes ‘wishful thinking’ as actors and their contributory behaviors are not sufficiently defined. In our case in Cambodia, Indigenous People had not yet experienced forums for ongoing participation in decision- making about how to fulfill Indigenous rights to quality education. Understanding that capacities are distributed across a social network of actors (Jones, 2011) means that we needed to make visible all who would contribute positively or negatively to the planned change. This supports accountability for their involvement and contributes to strengthening community capacity for civic participation and their right to self-determination. The Collaborative Purpose of the ToC The government plan for MLE in Cambodia was produced through the collaboration of multiple actors. The planning documents conveyed that national and provincial government offices and international development partners saw themselves as the primary duty bearers and most instrumental agents of changes. Indigenous children and their parents were seen as primary beneficiaries as well as rights holders. MLE teachers were seen as intermediaries, having the primary responsibility to deliver provided curriculum according to a prescribed ratio of Indigenous and non-Indigenous languages at each grade level. Our approach to evaluating the plan included Indigenous children, parents, village leaders, and Indigenous language groups, not only as beneficiaries, but also as significant influencers of the change process. We perceived these rights holders as potent contributors to whether intended targets were reached. This approach led to the identification of actors and activities that had not been identified in the original government plan. Importantly, the explicit government purpose of the MLE initiative to ease young Indigenous children’s transition to education in the (non-Indigenous) national language was questioned by Indigenous actors and rights holders, who expressed a primary goal of preserving Indigenous languages and securing education primarily in Indigenous languages for twice the length of time afforded by the government plan. As we have argued, inclusion of a broad array of actors and activities that may not be explicit in simplified, institution-focused ToCs can uncover unwarranted assumptions and interactions and enable the production of a context-specific, data- Focusing on Actors in Context-Specific, Data-Informed Theories of Change 33 informed ToC that is more complex, comprehensive, and useful for future planning. It is important to ensure that a ToC is not used as a tool to legitimize managerialism and oppression. Tools for accountability require users to challenge assumptions (Ramalingam, 2013). Uncovering and challenging implicit assumptions and ideas about how change will happen requires varied perspectives and trust. Our evaluation process brought diverse actors together in what we called “discovery workshops” to triangulate and synthesize data obtained through prior focus groups and interviews. These multi-stakeholder gatherings, bringing Indigenous parents and hamlet leaders and representatives of Indigenous Peoples’ Organizations together with educators and senior government policy makers, were unique in the Cambodian education context. Authentic and often courageous participation was enabled in part by participants’ prior engagement in interviews and focus groups in their own everyday settings, building trust in the process and confidence in expressing their views. Fixed and Linear or Iterative and Circular Change Process ToCs, if used well, can reflect and support iteration, learning, and adaptive practices. Iteration (the repetition and review of a process) and working with social feedback systems are strategies for working with complexity instead of demanding compliance to what implementers of a planned change knew at the start (Ramalingam, 2013). Schön (1991) distinguished between espoused theories (theories we believe that we work from) and theories of use (theories that we actually put into practice). Well- designed ToCs can effectively support the process of testing espoused theories (Schön, 1991), where not only success and failure, but also belief systems and assumptions are explored. In our Cambodian case example, the evaluation process uncovered new information about actors, activities and relationships in the system. External processes also affect the ToC, with changes in positions of power, uncertainty of funding sources, and the inclusion and exclusion in decision-making, of Indigenous People and organizations that represent them. The ToC as Being Context-Based or Borrowing from Other Contexts Learning, knowledge, and change are situated in a context (Lave & Wenger, 1991). Solutions are rarely generally applicable across social contexts, as humans are nested in specific relationships within a particular ecological system. Systems thinking, which underlies an actor-focused approach to ToCs, ensures that people in the system—their beliefs, boundaries and perspectives—are visible and engaged (Williams & Hummelbrunner, 2010). For managers, program implementers, and community members, this can provide clarity to better support unique roles. Interventions in a social ecosystem therefore need to recognize social and political aspects of change and not only the technical aspects. In our evaluation of the MLE plan in Cambodia, we explored who the key actors were at all levels of the teaching and learning system, including children, and what roles they played. Both formally and informally engaged actors were included in our actor- focused ToC; not only the most powerful or visible, but also those with less power, authority, and visibility. In network theory, Davies (2004) contends that sustainable development practice requires actors working in parallel. There is an interdependency and distributed capacity in each local context to support relevant and sustainable quality basic education. In contexts where actors have been sufficiently recognized and involved in an authentic way, it may be useful to deploy a theory-driven ToC. This is because learning 34 Global Education Review 7 (1) from theory could help inform and frame the work based on experience in other places. However, there is a growing skepticism about the concept of “best practices” so often visible in program logic. Aiming for a “best practice” demonstrated as effective in an external context and from an external set of values will not produce a community-engaged, realistic understanding of how change can happen in a specific context and likely barriers to success of a planned change. If context is taken seriously, the concept of “best practices” needs to be replaced with “best fit” (Ramalingam, Laric, & Primrose, 2014). This requires support for negotiated understandings of community-fitting goals and strategies and multiple viewpoints on assessments of progress. Externally defined best practices might help to inform the negotiation, but should not define or constrain it. Responsive management recognizes context-specificity and engages local actors in co-creating context- specific designs and learning. In describing efforts to do effective digital development differently, Prieto-Martin, Hernandez, and Ramalingam (2017) suggest drawing upon different sources for learning: contextual, evidential (from theory/others), and evaluative (from own experience gathered in the local context). The ToCs we produced for the evaluation in Cambodia drew upon these three sources. The ToC as a Heuristic Tool or a Positivist Reality A ToC can be yet another managerial tool for reinforcing hierarchical roles and reinscribing vertical authority in line with a dominant group’s view of reality. In this article, we emphasize the meaning-making opportunity that gathering data for constructing one or several different kinds of ToCs can provide. However, even concrete actor-focused ToCs such as we propose for complex educational interventions cannot, in themselves, support learning and adaptive management. In our experience, learning processes need key actors, including intended beneficiaries of the intervention, coming together to form relationships of trust and constructive communication for negotiation and reflection on practice (Shutt, 2016). This reflection needs to consider not only what happened but assumptions and framings of problems based on the experience—so called double loop learning (Schön, 1991; Valters, Cummings, & Nixon, 2016). Participatory information-gathering approaches (e.g., OH), and skilled facilitation in the languages of local actors can promote meaningful engagement and negotiations that draw out the often differing realities of key stakeholders in ways that are relevant to the vision, goals, strategies, activities, and intended outcomes of a planned education intervention. With this interpersonally engaged foundation, it will become meaningful to communally create an actor-focused ToC and track behavior mapped out in the ToC. Conclusion In complex situations where equity and social justice are sought after goals, such as the drive for greater inclusion of Indigenous children in quality basic education, we must search for win-win-win solutions for people, ecologies, and economies (Ramalingam, 2017). ToCs can potentially be useful to support education reforms, but we must be conscious of how the ToCs can shape and limit or open up the way we think and work in practice. Multiple forms of ToC can be useful as long as users and commissioners of ToCs provide timely and authentic opportunities for participation by local actors, acknowledging the ecosystems of various actors, and using the tool as an iterative, heuristic tool for learning, collaboration, and accountability. All of the actors in a system, Focusing on Actors in Context-Specific, Data-Informed Theories of Change 35 including duty-bearers and rights holders, are obligated and needed to make the system work, and must be accounted for in assessing the potential to successfully implement a planned change and evaluating outcomes. In this article, we have proposed actor- focused ToCs as an especially suitable type of cultural tool for learning and evaluating change processes; one that potentially encourages recognition of less visible or previously excluded actors and that maps pathways for their engagement with other actors in a change process. Actor-focused ToCs can challenge hegemonic approaches to change and strongly support win-win-win solutions called for in complex situations. In the MLE context used to illustrate our discussion of ToCs, less visible actors included Indigenous children and their parents or guardians, Indigenous community groups and language organizations, and national Indigenous Peoples’ Organizations. These less visible actors had context-specific knowledge about people, economies, and ecologies connected to their context, all potentially relevant for Indigenous education. As Hinton and Groves (2004) contend, “the challenge of political participation is not only a question of who is sitting around the table, but of whether the table even exists, and whether the language and terms of debate are accessible to those whose voices need to be heard” (p.12). An actor- focused ToC may be one avenue for enabling excluded or oppressed participants in an education system to assume meaningful roles in creative, context-fitting, community-paced, and community-based social change. An actor- focused ToC can hold governments and institutions accountable to inclusive, post- colonial approaches to education reform in order to fulfill children’s rights to meaningful basic education. References Ball, J. (2004). As if Indigenous knowledge and communities mattered: Transformative education in First Nations communities in Canada. American Indian Quarterly, 28(3/4), 454-479. Ball, J. (2005). “Nothing about us without us:” Restorative research partnerships involving Indigenous children and communities in Canada. In A. Farrell (Ed.), Exploring ethical research with children (pp. 81-96). Maidenhead, UK: Open University Press/McGraw-Hill Education. Ball, J., & Smith, M. (2019). Independent Evaluation of the Multilingual Education National Action Plan in Cambodia. Evaluation Report. Phnom Penh: UNICEF Cambodia Country Office. https://www.unicef.org/cambodia/media/1061/ file/2019-Cambodia%20MENAP- Evaluation%20Report-Volume-I.pdf%20.pdf Battiste, M. (2013). Decolonizing education: Nourishing the learning spirit. Saskatoon, Canada: Purich Publishing. Carden, F., & Earl, S. (2007). Infusing evaluative thinking as process use: The case of the International Development Research Centre (IDRC). New Directions for Evaluation, 2007(116), 61-73. Conlin, S., & Stirrat, R. L. (2008). Current challenges in development evaluation. Evaluation, 14(2), 193-208. Cummins, J. (2000). Language, power, and pedagogy: Bilingual children in the crossfire (Vol. 23). Multilingual Matters. Dahlstedt, M. & Olson, M. (2013). Utbildning, demokrati, medborgarskap. Malmö: Gleerups Utbildning. Davies, R. (2004). Scale, complexity and the representation of theories of change: Part I. Evaluation 10 (1), 101-121. https://www.unicef.org/cambodia/media/1061/file/2019-Cambodia%20MENAP-Evaluation%20Report-Volume-I.pdf%20.pdf https://www.unicef.org/cambodia/media/1061/file/2019-Cambodia%20MENAP-Evaluation%20Report-Volume-I.pdf%20.pdf https://www.unicef.org/cambodia/media/1061/file/2019-Cambodia%20MENAP-Evaluation%20Report-Volume-I.pdf%20.pdf 36 Global Education Review 7 (1) Davies, R. (2005). Scale, complexity and the representation of theories of change: Part II. Evaluation 11 (2), 133-149. Davies, R. (2018). Representing theories of change: A technical challenge with evaluation consequences. London: CEDIL Inception Paper 15. Earl, S., Carden, F., & Smutylo, T. (2001). Outcome mapping: Building learning and reflection into development programs. Ottawa: International Development Research Centre. Retrieved from http://www.outcomemapping.ca/resource/om- manual Funnell, S. C., & Rogers, P. J. (2011). Purposeful program theory: Effective use of theories of change and logic models. Vol. 31. San Francisco: John Wiley & Sons. Groves, L., & Hinton, R. (Eds). (2004). Inclusive aid: Changing power and relationships in international development. London: Earthscan. Heyman, J., & Cassola, A. (2012). Lessons in education equality: Successful approaches to intractable problems around the world. Oxford: Oxford University Press. Hill, J. C., & Stairs, A.H. (2002). Indigenous education: Ways of knowing, thinking, and doing. McGill Journal of Education, 37 (3), 281- 468. Jones, H. (2011). Taking responsibility for complexity: How implementation can achieve results in the face of complex problems. Working Paper 330. London: Overseas Development Institute. Kurtz, C. F., & Snowden, D. J. (2003). The new dynamics of strategy: Sense-making in a complex and complicated world. IBM Systems Journal, 42(3), 462-483. Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation in communities of practice. Cambridge: Cambridge University Press. McGregor, D. (2004). Coming full circle: Indigenous knowledge, environment, and our future. American Indian Quarterly, 28 (3/4), Special Issue: The recovery of Indigenous knowledge. 385-410. Patton, M. Q. (2008). Utilization-focused evaluation. 4th edition. Thousand Oaks: Sage. Prieto-Martin, P., Faith, B., Hernandez, K., & Ramalingam, B. (2017). Doing Digital Development Differently: lessons in adaptive management from technology for governance initiatives in Kenya. Making All Voices Count Research Report, October. Brighton: Institute of Development Studies. Ramalingam, B. (2013). Aid on the edge of chaos: Rethinking international cooperation in a complex world. Oxford: Oxford University Press. Ramalingam, B., Laric, M., & Primrose, J. (2014). From best practice to best fit: understanding and navigating wicked problems in international development. Overseas Development Report. London: Overseas Development Institute, 1-44. Rogers, P. (2014). Theory of change: Methodological briefs-impact evaluation, 2 (No. innpub747). Royal Government of Cambodia, Ministry of Education, Youth and Sport (2015). Multilingual Education National Action Plan (MENAP) 2015- 2018. Phnom Penh: Author. Schö̈n, D.A. (1991). The reflective practitioner: How professionals think in action. London: Basic Books. Shutt, C. (2016). Towards an alternative development management paradigm. Report 2016:07 to EBA [Expertgruppen för biståndsanalys]. Retrieved from http://eba.se/wp- content/uploads/2016/08/Rapport2016_07_we bb.pdf Smith, M. (2017). Whose story? Exploring communicative practices among international http://www.outcomemapping.ca/resource/om-manual http://www.outcomemapping.ca/resource/om-manual http://eba.se/wp-content/uploads/2016/08/Rapport2016_07_webb.pdf http://eba.se/wp-content/uploads/2016/08/Rapport2016_07_webb.pdf http://eba.se/wp-content/uploads/2016/08/Rapport2016_07_webb.pdf Focusing on Actors in Context-Specific, Data-Informed Theories of Change 37 development organisations using learning approaches designed for complex situations (Unpublished Masters Thesis). Skövde University, Skövde, Sweden. Thaler, R. (2009). Nudge: Improving decisions about health, wealth, and happiness. New York: Penguin. UNESCO (authored by Malone, S.) (2018). MTB MLE resource kit: Including the excluded– Promoting multilingual education. Bangkok: Author. Valters, C. (2015). Theories of change. Time for a radical approach to learning in development. Overseas Development Report, March. London: Overseas Development Institute. Valters, C., Cummings, C., & Nixon, H. (2016). Putting learning at the centre: Adaptive development programming in practice. Overseas Development Report, March. London: Overseas Development Institute. Vogel, I. (2012). Review of the use of “Theory of Change” in international development. London, UK: Department for International Development (DFID). Retrieved from: http://www.theoryofchange.org/pdf/DFID_ToC _Review_VogelV7.pdf Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. New York: Cambridge University Press. Wertsch, J. V. (1998). Mind as action. Oxford: Oxford University Press. Wilson-Grau, R. (2018). Outcome harvesting: Principles, steps, and evaluation applications. Charlotte, NC: Information Age Publishing. Williams, B., & Hummelbrunner, R. (2010). Systems concepts in action: a practitioner's toolkit. Stanford: Stanford University Press. Wong, K. M., & Benson, C. (2019). Language as gatekeeper for equitable education: Multilingual education in Cambodia. In J. Gube & F. Gao (Eds.), Education, ethnicity and equity in the multilingual Asian context (pp. 69-86). Springer, Singapore. About the Authors Mariam Smith is an independent consultant (Learning Loop Inc), supporting program teams’ and community members’ learning through program design, meaningful monitoring systems, and evaluation. She specializes in actor- centered methodologies including Outcome Mapping and Outcome Harvesting. Most of her work has been within the field of international development and education in Scandinavia and in Southeast Asia, where she lived for 15 years working with Indigenous rights and education. She speaks several dominant and Indigenous languages, and has conducted international evaluations of government and non-government initiatives to promote children’s learning in their home language. Jessica Ball is a professor at the University of Victoria (Canada). She has developed and taught courses about early childhood health and development and evaluation research methods at universities in Canada, the United States, and several countries in Asia. She developed, delivered, and evaluated programs for numerous international organizations and government ministries of education. Her research is often partnered with communities, and focuses on determinants of marginalization and promising practices for inclusion of migrant, minoritized, and Indigenous children and families. http://www.theoryofchange.org/pdf/DFID_ToC_Review_VogelV7.pdf http://www.theoryofchange.org/pdf/DFID_ToC_Review_VogelV7.pdf 38 Global Education Review 7 (1) Appendix Figure 1: A generic ToC: Reconstructed theory of change deduced from the MENAP 2015-2018 (MT=mother tongue, M&E=monitoring and evaluation, PED=Primary Education Department, ECED=Early Childhood Education Department) Focusing on Actors in Context-Specific, Data-Informed Theories of Change 39 Figure 2: Stakeholder mapping/network analysis and an actor-focused theory of change. (MoEYS=Ministry of Education Youth and Sports, POE=Provincial Office of Education, DOE=District Office of Education, PTTC=Provincial Teacher Training College, IPO=Indigenous People Organizations) 40 Global Education Review 7 (1) Figure 3: Analysis of strategies compared to components of UNESCO best practice in MLE. (MoEYS=Ministry of Education Youth and Sport, ECED=Early Childhood Education Department, PED=Primary Education Department)