Gateways: International Journal of Community Research and Engagement Vol. 12, No. 2 December 2019 © 2019 by the author(s). This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International (CC BY 4.0) License (https:// creativecommons.org/licenses/ by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license. Citation: Busch, MD, Jean- Baptiste, E, Person, PF, Vaughn, LM. 2019. Activating social change together: A qualitative synthesis of collaborative change research, evaluation and design literature. Gateways: International Journal of Community Research and Engagement, 12:2, Article ID 6693. http:dx.doi.org/10.5130/ ijcre.v12i2.6693 ISSN 1836-3393 | Published by UTS ePRESS | http://ijcre. epress.lib.uts.edu.au RESEARCH ARTICLE Activating social change together: A qualitative synthesis of collaborative change research, evaluation and design literature Melida D Busch1, Elizabeth Jean-Baptiste2, Pamela F Person1, Lisa M Vaughn3 1 Education Innovations & Community Partnerships, College of Education, Criminal Justice, and Human Services, University of Cincinnati, 2600 Clifton Avenue, Cincinnati OH 45220 2 College of Education, Criminal Justice, and Human Services, University of Cincinnati, 2600 Clifton Avenue, Cincinnati OH 45220 3 Pediatrics, Cincinnati Children’s Hospital Medical Center/University of Cincinnati College of Medicine, Joint appointment, College of Education, Criminal Justice, and Human Services, Educational and Community-Based Action Research, University of Cincinnati, 2600 Clifton Avenue, Cincinnati OH 45220 Corresponding author: Melida D Busch; melida@knowful.org DOI: http:dx.doi.org/10.5130/ijcre.v12i2.6693 Article history: Received 24/07/2019; Revised 14/11/2019; Accepted 25/11/2019; Published 20/12/2019. Abstract Researchers, evaluators and designers from an array of academic disciplines and industry sectors are turning to participatory approaches as they seek to understand and address complex social problems. We refer to participatory approaches that collaboratively engage/ partner with stakeholders in knowledge creation/problem solving for action/social change outcomes as collaborative change research, evaluation and design (CCRED). We further frame CCRED practitioners by their desire to move beyond knowledge creation for its own sake to implementation of new knowledge as a tool for social change. In March and May of 2018, we conducted a literature search of multiple discipline-specific databases seeking collaborative, change-oriented scholarly publications. The search was limited to include peer- reviewed journal articles, with English language abstracts available, published in the last five years. The search resulted in 526 citations, 236 of which met inclusion criteria. Though the search was limited to English abstracts, all major geographic regions (North America, Europe, DECLARATION OF CONFLICTING INTEREST The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. FUNDING The author(s) received no financial support for the research, authorship, and/or publication of this article. 1 https://creativecommons.org/licenses/by/4.0/ https://creativecommons.org/licenses/by/4.0/ https://creativecommons.org/licenses/by/4.0/ http:dx.doi.org/10.5130/ijcre.v12i2.6693 http:dx.doi.org/10.5130/ijcre.v12i2.6693 http://ijcre.epress.lib.uts.edu.au http://ijcre.epress.lib.uts.edu.au mailto:melida@knowful.org http:dx.doi.org/10.5130/ijcre.v12i2.6693 Latin America/Caribbean, APAC, Africa and the Middle East) were represented within the results, although many articles did not state a specific region. Of those identified, most studies were located in North America, with the Middle East having only one identified study. We followed a qualitative thematic synthesis process to examine the abstracts of peer-reviewed articles to identify practices that transcend individual disciplines, sectors and contexts to achieve collaborative change. We surveyed the terminology used to describe CCRED, setting, content/topic of study, type of collaboration, and related benefits/outcomes in order to discern the words used to designate collaboration, the frameworks, tools and methods employed, and the presence of action, evaluation or outcomes. Forty-three percent of the reviewed articles fell broadly within the social sciences, followed by 26 percent in education and 25 percent in health/medicine. In terms of participants and/ or collaborators in the articles reviewed, the vast majority of the 236 articles (86%) described participants, that is, those who the research was about or from whom data was collected. In contrast to participants, partners/collaborators (n=32; 14%) were individuals or groups who participated in the design or implementation of the collaborative change effort described. In terms of the goal for collaboration and/or for doing the work, the most frequently used terminology related to some aspect of engagement and empowerment. Common descriptors for the work itself were ‘social change’ (n=74; 31%), ‘action’ (n=33; 14%), ‘collaborative or participatory research/practice’ (n=13; 6%), ‘transformation’ (n=13; 6%) and ‘community engagement’ (n=10; 4%). Of the 236 articles that mentioned a specific framework or approach, the three most common were some variation of Participatory Action Research (n=30; 50%), Action Research (n=40; 16.9%) or Community-Based Participatory Research (n=17; 7.2%). Approximately a third of the 236 articles did not mention a specific method or tool in the abstract. The most commonly cited method/tool (n=30; 12.7%) was some variation of an arts-based method followed by interviews (n=18; 7.6%), case study (n=16; 6.7%), or an ethnographic-related method (n=14; 5.9%). While some articles implied action or change, only 14 of the 236 articles (6%) stated a specific action or outcome. Most often, the changes described were: the creation or modification of a model, method, process, framework or protocol (n=9; 4%), quality improvement, policy change and social change (n=8; 3%), or modifications to education/training methods and materials (n=5; 2%). The infrequent use of collaboration as a descriptor of partner engagement, coupled with few reported findings of measurable change, raises questions about the nature of CCRED. It appears that conducting CCRED is as complex an undertaking as the problems that the work is attempting to address. Keywords: collaborative change, social change, qualitative synthesis, literature review, collaborative research, collaborative evaluation, collaborative design Introduction Researchers, evaluators and designers from an array of academic disciplines and industry sectors are turning to participatory approaches as they seek to understand and address complex social problems. Participatory approaches expand the tools available to investigate multifaceted issues; they differ from traditional methods by intentionally integrating local community norms and knowledge into discovery processes to develop more relevant research questions, inform research interpretation, share findings, innovate, and foster sustained change Busch, Jean-Baptiste, Person, Vaughn Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 2 (Balazs & Morello-Frosch 2013; Cargo & Mercer 2008; Chang et al. 2013; Person et al. 2016). These contextual and relationship-driven participatory approaches strive for different outcomes and new understandings that represent lived experience and value the standpoints, expertise and direct involvement of stakeholders and end users of research, evaluation and design. Such approaches offer a research-to-action imperative that bridges the common gap between research and practice (Cargo & Mercer 2008). Approaches such as community-based participatory research, empowerment evaluation, human-centred design, and citizen science engage stakeholders with differing backgrounds or expertise as collaborative partners in knowledge creation and problem solving towards positive social change. We have coined the umbrella term – collaborative change research, evaluation, and design (CCRED) – to collectively refer to the array of participatory approaches utilised by researchers, evaluators and designers who work in this ‘bridge’ space of research and practice for positive social change. Background Globally, institutions of higher education are facing serious challenges as to whether they are preparing students who are able to effect positive social change and fulfil social/community/ global responsibilities (Gamoran 2018; Hayter & Cahoy 2018). Gamoran (2018) asserts that ‘institutions that turn their attention to serving the public good may be best poised to thrive and deliver lasting value’. Institutionally, much knowledge has been generated with the intention to solve complex problems; however, knowledge alone is inadequate to bring about sustainable social change (Lotz-Sisitka et al. 2015). Striving to meet such challenges requires ‘action-oriented capabilities’ (Lotz-Sisitka et al. 2015) and will likely require institutions to rethink and re-tool traditional approaches to research, evaluation and design, which more often than not are deficit-based and siloed, and compounded by a lack of collaboration across sectors and disciplines. Furthermore, traditional approaches to research, evaluation and design fail to have social impact because they are ‘not sufficiently timely, relevant, or accessible’ (Gamoran 2018). Those researchers, educators and designers who seek to utilise their efforts to contribute actively to society are attempting to employ participatory methods in order to bridge the gap between contributing to an inert body of knowledge and implementing this knowledge to achieve positive real-world change. People who work in the CCRED arena focus on resolving complex social issues, such as improving public health, achieving equitable public education and preserving the environment, to achieve better life conditions. Social change is the aim. In their influential case study on successful social change efforts, Kania and Kramer (2011) articulated a social change operating framework, which they named Collective Impact (CI). CI seeks to shift problem solving of complex community-based issues away from efforts by multiple practitioners working in isolation from one another towards engaging a collective of practitioners and affected stakeholders to collaboratively mobilise their resources towards shared learning and solution design and evaluation to achieve agreed upon goals and bring about large-scale systemic change. The CI framework is driven by a neutral backbone organisation, through a network of relationships between organisations, their common vision, constant communications, leveraged and mutually reinforcing activities, and shared measurement of change (Cabaj & Weaver 2016; Kania & Kramer 2011). With a goal of creating knowledge and structures that improve life’s conditions, CI is one example of a participatory approach that has emerged in the last 10 years, and one with which we have been engaged through our work at the University of Cincinnati. In 2018, the National Science Foundation granted the University of Cincinnati (UC) an INCLUDES (NSF #1812795) award that aims to ‘broaden participation projects’, ‘catalyze Activating social change together: A qualitative synthesis of collaborative change research, evaluation and design literature Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 3 the STEM enterprise to collaboratively work for inclusive change’ and ‘support scholars engaged in broadening participation research’. A review of the literature was one of the grant’s initial steps in understanding the scope of language and practice being used across disciplines, seeking to create a space within which to unify and expand collaboration. During a convening of local university and community-based practitioners of participatory change methods, where an early version of the themes in this synthesis were discussed, multiple layers of difference became apparent. In addition to variance in academic disciplines represented, within disciplinary groups there were differences in understanding, terminology and approaches used by people who identified as researchers, evaluators or designers. Our experience with CI in a variety of social change endeavours over multiple years provides context for this synthesis review. The CI operating framework implies key drivers for resolving social issues that need to be made explicit, for example, systems perspective, cross-sector partnerships, engaged stakeholders, shared understanding, transparent power and known collaborative change practices. In our CI work we found ourselves engaging with teams of community-based, cross-sector and cross-discipline partners and stakeholders – including researchers, evaluators and designers – who possessed critically important and diverse perspectives, but who rarely came together equipped with commonly held terminology, principles, tools, methods or measures to effectively create change. To begin addressing the need to equip practitioners who hold differing expertise and roles, and who employ a wide range of designs, methodologies and methods, we sought to understand the current field of participatory and collaborative change research, evaluation and design across industry sectors and academic disciplines, with the intent of providing a synthesis that would bridge CCRED approaches and their practice. We now discuss three broad areas of CCRED – participatory research, participatory evaluation and participatory design. Participatory research Reason and Torbert (2001) argue that the purpose of research has evolved. They describe the ‘action turn’ phenomenon in social science research, which has extended the purpose of research beyond that of knowledge generation for the sake of science. Research is also conducted to ‘forge a more direct link between intellectual knowledge and moment-to- moment personal and social action, so that inquiry contributes directly to the flourishing of human persons, their communities, and the ecosystems of which they are part’ (Reason & Torbert 2001, p. 6). This more participatory research-to-action turn is a ‘radical shift’ from the unquestioned traditional research grounding in empirical positivism. In contrast to traditional approaches that conduct research on persons, an action-oriented approach suggests the importance of practical knowing and a collaborative approach to inquiry – research with people – ‘including them both in the questioning and sensemaking that informs the research, and in the action which is the focus of the research’ (Reason & Torbert 2001, p. 10). The action turn phenomenon can be seen over the last decade as researchers have increasingly sought to engage service users, consumers, patients, families, youth and other individuals normally considered passive subject participants in more active, influential roles, including as research collaborators and decision-makers (Brett et al. 2014; Cabassa et al. 2017; Concannon et al. 2014; Domecq et al. 2014; Ellis & Kass 2017; Nilsen et al. 2006; Shen et al. 2017; Shippee et al. 2015). This shift in participant roles begs questions about what level of collaboration researchers should seek. Regardless of the level of engagement – minimal Busch, Jean-Baptiste, Person, Vaughn Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 4 (e.g. focus groups), medium (e.g. advisory) or high (e.g. shared decision-making), successful researcher–stakeholder collaboration is inclusive and reflects shared leadership, trust and authentic interaction around solving the issue at hand (Cashman et al. 2008; Newman et al. 2011; NIH 2011; Salimi et al. 2012; Wallerstein et al. 2018). Elevated engagement with the community voice – local knowledge, norms and desires, and socio-cultural, historical, political, economic, and ecological contexts of the people affected – is now integral to knowledge discovery and sustainable social change in participatory research (Ganz 2010; Rosenthal et al. 2014; Vaughn et al. 2017; Wallerstein et al. 2018; Weisbord 2012). Participatory evaluation Collaborative and participatory approaches to evaluation seek to engage those who have the greatest stake in a program’s effectiveness – from a program’s beneficiaries to its funders – in all phases of evaluation development and implementation (Fetterman et al. 2017; O’Sullivan 2012; Patton 1997). Participatory evaluation broadly represents several collaborative evaluation approaches, including empowerment evaluation and transformative evaluation (Cooper 2014; Cousins & Whitmore 1998; Fetterman et al. 2017; Fetterman & Wandersman 2005; Mertens 2008). Focusing on the stakeholder involvement aspects and the essential features of participatory evaluation rather than the specific type, the American Evaluation Association, for instance, combines participatory evaluation approaches in their Collaborative, Participatory, and Empowerment Evaluation Topical Interest Group (Fetterman et al. 2014). Such participant-oriented evaluation approaches range on a continuum in three ways: (1) control of decision making, ranging from evaluator to stakeholder; (2) depth of participation, ranging from consultation to deep participation; and (3) diversity of stakeholders invited to participate, ranging from limited to diverse. In contrast to more traditional approaches to evaluation, collaborative and participatory evaluation ensures that the evaluation is focused on topics and questions that are relevant to the community’s context; generates knowledge that is reflection-oriented and can be practically applied to improve program performance; honours local talent and expertise and empowers the local community to control decision making about things that affect them; builds expertise and capacity for future leadership and growth; strengthens community resources and networks; and supports sustainability (Cooper 2014; Cousins & Whitmore 1998; Fetterman et al. 2017; Fetterman & Wandersman 2005; Mertens 2008). Participatory evaluation differs from conventional evaluation in that it shifts the locus of power from funders, program managers and outside experts and instead shares it with multiple stakeholders, including community members, staff and others engaged in or affected by a program in order to ‘acknowledge and elevate the perspectives, voices, and decisions of the least powerful and the most affected stakeholders’ (Rossman 2000). Rather than serving as outside experts, lead evaluators function as coach or facilitator of the evaluation (Zukoski & Luluquisen 2002), with goals of strengthening evaluation designs, optimising data collection and analysis, and improving stakeholder use of the evaluation results (O’Sullivan 2012). The key distinction among these participatory types of evaluation is the degree of control between evaluators and stakeholders. In collaborative evaluation, evaluators oversee the evaluation with stakeholders, giving input at each stage of the evaluation (O’Sullivan 2012). In participatory evaluation, evaluators and stakeholders engage in joint control of the evaluation. Often, ‘control begins with the evaluator but is divested to program community members over time and with experience’ (Cousins, Whitmore & Shulha 2013, p. 14). Empowerment evaluation views stakeholders as the primary controllers of the evaluation, with evaluators Activating social change together: A qualitative synthesis of collaborative change research, evaluation and design literature Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 5 serving as critical friends and coaches to guide the process (Fetterman & Wandersman 2005). Transformative evaluation emphasises social justice and the inclusion of marginalised groups and communities who have traditionally been excluded from traditional evaluations (Cooper 2014; Mertens 1999, 2008). There is debate in the evaluation field as to whether or not stark distinctions between these approaches are important to maintain, with Cousins and Chouinard (2012) arguing that context and guiding principles, rather than a predetermined purpose or form of collaboration, should drive decisions about the most appropriate type of collaborative evaluation to use. Most evaluation contexts, they say, are complex and ‘messy’. They are ‘unpredictable and in constant flux … there is no right answer; rather, over time, patterns can be discerned, and a path forward emerges. In complex contexts, there are many opportunities for creativity and innovation’ (Cousins & Chouinard 2012, p. 17). Participatory design The design field is remarkably diverse, drawing theorists and practitioners from a wide range of disciplines and epistemological roots. Johansson‐Sköldberg, Woodilla and Çetinkaya (2013) have identified influential thinkers and practitioners in the design field who have backgrounds in economics, political science, philosophy, music, art history, architecture, and business management, as well as design. Their research on design and design(erly) thinking discourse identifies themes in academic (scholarly) discourse that can be categorised as: creating artifacts, reflexive practice, problem-solving activity, way of reasoning or sense- making, and creating meaning. Within management (applied) discourse, they identify themes that can be categorised as: a way of working, a requisite skill for managers (across disciplines and industries) and part of management theory (including change management). Design(erly) thinking, they conclude, ‘is not one but many’ approaches in ‘an ongoing design practice, a reality that is not a discrete and coherent practice, and is far from standardized, but is nevertheless the basis for generalizations, descriptions and theories’ ( Johansson‐Sköldberg, Woodilla & Çetinkaya 2013, p. 131). Buchanan’s 1992 breakthrough article, which introduced a new way to contextualise design solutions within overlapping spaces, rather than creating design solutions in sequential steps, positioned design thinking as a participatory field especially well suited to addressing complex social issues, called ‘wicked problems’. Wicked problems are ‘a class of social systems problems with a fundamental indeterminacy without a single solution and where much creativity is needed to find solutions’ ( Johansson-Skӧldberg, Woodilla & Çetinkaya 2013, p. 125). Brown and Wyatt (2010) describe design thinking as ‘deeply human’. They explain that design thinking can be conceived of in terms of three overlapping spaces: inspiration (problem/ opportunity), ideation (generating, developing and testing ideas) and implementation (prototyping, refinement, communication for launch). The process involves inclusive teams and is experiential, participatory and interdisciplinary, with ‘empathy for people and for disciplines beyond one’s own’ a key concept. In the context of inspiration, ‘design thinkers become embedded in the lives of the people they are designing for … relying on local partners and ‘cultural guides’ to help inform and continue to participate in the processes of ideation and implementation (Brown & Wyatt 2010, p. 33). Busch, Jean-Baptiste, Person, Vaughn Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 6 Collaborative Change Research, Evaluation and Design (CCRED) These participatory and action-oriented approaches to research, evaluation and design promote bidirectional communication, enhanced trust, shared leadership and ownership, and mutual benefits between academics and communities in the research process. Furthermore, the quality and rigour of research, evaluation and design are improved by participant engagement, contextually relevant questions, externally valid data interpretation, and culturally valid measurement instruments and techniques (Balazs & Morello-Frosch 2013; Buchanan, Miller & Wallerstein 2007; Diaz, Spears Johnson & Arcury 2015). The results of such approaches are often more widely disseminated (Chen, Weiss & Nicholson 2010) and have greater potential for real-world impact and sustainable actions. There are many commonalities, shared practices and guiding principles across these approaches, yet siloes remain intact, with fierce kinship to discipline and approach-specific nomenclature and guidelines. There is an opportunity for disciplines to benefit from shared definitions, principles, tools/methods, and measures of outcomes in terms of change, impact and action. CCRED intentionally invites partners to work on complex, transdisciplinary issues from multiple perspectives. The purpose of this synthesis review was to systematically examine peer- reviewed articles to identify effective practices that transcend individual disciplines, sectors and contexts to achieve CCRED. We examined the terminology and titles used to describe CCRED, details of the process, setting, content/topic of study, type of collaboration and related benefits/outcomes. Specifically, we aimed to answer these research questions: 1. What terminology is used for and how is collaboration defined within CCRED across disciplines? 2. What types of frameworks/approaches are being used within CCRED? 3. What are the methods and tools used in CCRED? 4. When change/action is discussed, what is the nature of the change/action in CCRED and how is it measured/evaluated? Method PEER-REVIEWED LITERATURE SEARCH STRATEGY We performed a Literature Search in March of 2018 and updated it in May of 2018 to include new and non-USA publications (Table 1). The search targeted multiple discipline-specific databases to gain the widest possible perspective of collaborative, change-oriented scholarly publications across the areas of research, evaluation and design. The following strategy was utilised in each database: (‘social change’ OR ‘change research’ OR ‘collaborative change’) AND (‘socially engaged’ OR ‘community engaged’ OR ‘community engagement’ OR ‘action research’ OR ‘community action’ OR ‘coproduction’ OR ‘design thinking’ OR ‘collaborative research’ OR ‘co-design’ OR ‘collaborative design’ OR ‘participatory design’ OR ‘peer researcher’ OR ‘co-researcher’ OR ‘participatory research’) AND (research OR design OR evaluation). SELECTION CRITERIA The search was limited to include peer-reviewed scholarly articles, with English language abstracts available, published in the last five years. Non peer-reviewed articles, conference proceedings, reports, theses/dissertations and book chapters were removed from the search Activating social change together: A qualitative synthesis of collaborative change research, evaluation and design literature Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 7 results. Post de-duplication of the 526 returned citations, 390 citations, including author(s), title, journal name, year of publication and abstract, were loaded into Covidence, a systematic review management system, for the team’s initial review. DATA SCREENING, ELIGIBILITY AND EXTRACTION Of the 390 publications in Covidence, 81 were removed for not meeting the initial requirements for coding, that is: (1) they were non peer-reviewed articles, conference proceedings, reports, theses/dissertations, or book chapters; (2) did not have an English abstract; or (3) were not ‘on topic’. See Figure 1 for a visual of the screening process. To begin the coding process, 50 of the remaining 309 citations were randomly selected for review. Table 1 Databases and number of citations Database Citations Art Full Text 7 Avery Index to Architectural Periodicals 1 BIOSIS Citation Index 7 Education Research Complete 131 Humanities Index 66 PsychInfo 106 PubMed 73 SocINDEX with Full Text 135 Figure 1 Article screening process Busch, Jean-Baptiste, Person, Vaughn Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 8 Based on this initial review, the team created an agreed-upon extraction database/codebook and definitions to be used to (1) further assess eligibility, ensuring that the selection criteria had been met, and (2) as the basis for summarising and analysing retained articles. The database included the following column categories for each publication: type of article; named participants/collaborators; terminology to describe collaborators; setting/context/geographic location; discipline; topic/area of concern; specific named frameworks/approaches; terminology of engagement/collaboration; specified tools/methods; action taken; and outcomes/reports of change. See Table 2 for a summary of agreed definitions, categories and codes. We reviewed our initial work and all four members of the team coded an additional 73 articles using discussion and consensus to resolve disagreements about coding. For the remaining 184 citations, the team then worked in three rounds of rotating pairs to code 60–65 articles per round. After each round of coding, we met as a team and reviewed our codes, resolving any disagreements through consensus, and sharing the finalised codes. For each of the records retained, examination of the abstracts (and full articles when details were unclear) and data synthesis were conducted. During this in-depth exploration and coding of the articles, an additional 73 articles were excluded as they did not meet the selection criteria. Post-review, a total of 236 articles included collaborative change efforts and were fully coded according to our review aims. Any uncertainties about how to code individual article information were discussed among the team members until consensus was reached. DATA SYNTHESIS Due to the nature of the review aims and the diversity of the literature retrieved, we conducted a qualitative thematic synthesis (Thomas & Harden 2008) of the data in contrast to a meta- analysis. We relied explicitly on the method’s functionality in ‘identifying key concepts from studies and translating them into one another’, allowing us to group concepts appearing across the abstracts reviewed, though these concepts were not articulated using precisely the same wording (Thomas & Harden 2008, p. 3). Across the 236 publications included in the review, patterns in the data were summarised inductively to examine and describe collaborative change efforts across research, evaluation and design. Table 2 Definitions of categories and codes Category Description Type of article Classification of article as research study, process, review or mixed Participants Group of people or community involved in the study (data collected from or about) Collaborators/true co- researchers Those who partnered in the design or implementation of the collaborative change effort Terminology used to describe collaborators Titles/names used for the collaborators (i.e. co- researcher and community health worker) Setting Context of project (i.e. neighbourhood, school) Country/geographical region City and/or country where project was conducted Activating social change together: A qualitative synthesis of collaborative change research, evaluation and design literature Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 9 Category Description Discipline Area of practice/knowledge/study Area of concern/topic Content area Framework/approach to collaborative change Theoretical perspective or orientation to collaborative change (i.e. action research, design thinking) Terminology of engagement/ collaboration Term(s) used to describe engagement/collaboration within project Methods Tools used to collect data or conduct collaborative change work (i.e. photovoice, interviews, case study, surveys, ethnodrama) Action/change Outcomes of project Results The 236 abstracts included in the review were coded as being focused on ‘Research’ (n=128; 54%), ‘Process’ (n=65; 28%), ‘Mixed’ (n=38; 16%) or ‘Review’ (n=5; 2%). ‘Research’ articles represented a study which yielded some type of conclusion, results or new knowledge. ‘Process’ articles described methods or design of a study or proposed a new model for further study based on previous research. Those categorised as ‘Mixed’ included any combination of the other categories, though the majority were a combination of ‘Research’ and ‘Process’ (n=34; 89.47%). ‘Review’ articles concentrated on systematic reviews of the literature regarding a particular aspect of collaborative change-oriented work. Representing over 150 distinct journals, the top six journals for publication across the 236 articles were: Journal of Community Practice (n=9; 3.8%); Action Research (n=9; 3.8%); Educational Action Research (n=8; 3.4%); American Journal of Community Psychology (n=6; 2.5%); Science and Engineering Ethics (n=6; 2.5%); and Global Public Health (n=5; 2%). Over 40% (n=105) of the 236 articles did not specify a geographic location for their work in the abstracts. Approximately 21% (n=50) of the articles specified that their collaborative change work was conducted in North America, primarily in the United States. The work in the remaining articles mostly occurred in Europe (n=35; 15%), Africa (n=21; 9%), Australia (n=14; 6%) and South America (n=12; 5%). Forty-three percent of the 236 articles fell broadly within the social sciences (i.e. psychology, sociology, social work, ecology, organisation development and community practice/ development) followed by 26% in education and 25% in health/medicine. Across all the articles, there was a cross-cutting interest in: (1) identity, community and inclusion (n=54; 23%); (2) participation and participatory processes (n=40; 17%); and (3) youth voice and focus (n=31; 13%). Representing the complexity and broad range of biopsychosocial, ecological and cultural change efforts across disciplines, there was a wide array of topics and areas of concern covered in the 236 reviewed articles. Specific topics within the social science articles ranged from human trafficking, immigration, violence prevention, climate change and intercultural relations to creativity and sexuality. Within education, article topics varied from higher education, leadership, science education, cooperative learning, disability education and literacy to educational inequities and racism. Within the health/medicine articles, specific topics Table 2 continued Busch, Jean-Baptiste, Person, Vaughn Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 10 included elder care/ageing, indigenous health, nutrition/food, disease prevention, physical activity and racial/ethnic disparities. TERMINOLOGY AND COLLABORATION IN CCRED In terms of participants and/or collaborators in the articles reviewed, the vast majority of the 236 articles (n=203; 86%) described participants, that is, those who the research was about or from whom data was collected. The most common descriptors for participants were ‘youth’, ‘students’ and ‘community members’. The descriptors varied widely, including Aboriginal children and young people, engineering students, South African grandmothers, families, junior high students, residents living in public housing, members of multi-ethnic communities, Mongolian teachers, college students with incarceration histories, First Nations peoples and violence prevention practitioners. Thirty-two of the 236 articles (n=32; 14%) named partners/ collaborators for their projects. In contrast to participants, partners/collaborators were individuals or groups who participated in the design or implementation of the collaborative change effort described. For instance, 18 older adults took a primary role as co-researchers to conduct interviews and participate in reflection meetings as part of a research project to develop age-friendly communities in Manchester, UK (Buffel 2019). In another article, Yup’ik elders, who were collaborators in the research effort, served as co-authors for the resulting publication (Ayunerak et al. 2014). The most common descriptor for collaborators was ‘community members’; other descriptors for collaborators included ‘stakeholders’ and ‘co-researchers’. Unique descriptors for collaborators included ‘co-creators’, ‘co-enquirers’, ‘insiders’, ‘cultural navigators’ and ‘change agents’. In describing collaboration, doing the actual work and/or the goal of the work in the 236 articles, the most frequently used terminology related to some aspect of engagement and empowerment. Terminology included ‘participation or participatory’, ‘collective’, ‘equity’, ‘inclusion’, ‘capacity’, ‘social justice’ and ‘democratic’. Common descriptors for the work itself were ‘social change’ (n=74; 31%) followed by ‘action’ (n=33; 14%), ‘collaborative or participatory research/practice’ (n=13; 6%), ‘transformation’ (n=13; 6%) and ‘community engagement’ (n=10; 4%). FRAMEWORKS AND APPROACHES WITHIN CCRED Across research, evaluation and design articles, the majority mentioned more than one specific framework or approach within the context of social change or community engagement. Of the 236 articles that mentioned a specific framework or approach, the most common framework/ approach, which was highlighted in 60 articles, was some variation of Participatory Action Research (PAR) (n=30; 50%), i.e. community-based, feminist, inclusive, arts-based, including Participatory Research (PR) (n=19; 31.7%), i.e. anthropological, educational, visual, and Youth Participatory Action Research (YPAR) (n=11; 18.3%). The second most common framework/approach described in the 236 articles was some variation of Action Research (n=40; 16.9%). Within the Action Research (AR) articles, specific types of AR included collaborative, group, critical utopian, ethnographic, reflective, systemic, service outcomes, and community, community-based or community-engaged. The third most common framework/ approach specified in the 236 articles was Community-Based Participatory Research (CBPR) (n=17; 7.2%) or a related variation (n=10; 4.2%), i.e. community-based participatory video, community-based arts, community-based service learning. Activating social change together: A qualitative synthesis of collaborative change research, evaluation and design literature Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 11 Although these frameworks and their variations were used broadly across all disciplines within the 236 articles, PAR and AR were most prevalent in education-related articles and CBPR was more commonly specified in health-related articles. Beyond PAR, AR and CBPR, as described above, other named frameworks within research-specific articles included social change, partnership, transformative, empowering, and advocacy-oriented research. Named frameworks within design-specific articles included design thinking, user-centred, human- centred, user-led, participatory, participatory research and public design, e.g. when discussing the use of user-centred design by neural engineers seeking to better understand the experiences and needs of potential end-users of the brain computer interface technologies (Sullivan et al. 2018). Within evaluation-specific articles, stated frameworks included participatory, inclusive, developmental, illuminative and action evaluation. For instance, van Winkelen (2016) used developmental evaluation methods to determine support resources in several communities of practice. METHODS AND TOOLS IN CCRED Approximately a third of the 236 articles (n=76; 32.2%) did not mention a specific method or tool in the abstract. Of those that did, the methods/tools varied considerably and/or used multiple methods in the same project. The most commonly mentioned method/tool (n=30; 12.7%) was some variation of an arts-based method (photovoice, participatory theatre, video, media posters, etc.), followed by interviews conducted either one-on-one or in a focus/ discussion group (n=18; 7.6%), case study (n=16; 6.7%) or an ethnographic-related method such as observation or ethnodrama (n=14; 5.9%). Table 3 contains a more detailed list of tools and methods used in the 236 articles. Table 3 Specified methods and tools Method/tool Number of articles (%) 2–3 example articles Detailed example/ description No method/tool specified 76 (32%) Miscellaneous (e.g. future conference, social network analysis, survey, community- based auditing, participatory appraisal, etc.) 41 (17%) Action research in nursing homes (Andersen & Bilfeldt 2016). Experiences in conducting participatory communication research for HIV prevention globally: Translating critical dialog into action through action media (Parker & Becker-Benton 2016). Participatory demographic scenarios addressing uncertainty and transformative change in Ethiopia (Tegegne et al. 2016). Using future workshops as a methodological tool in a joint action research project between care workers, residents at the nursing home, and researchers, this article details changes made to improve public elder care quality (Andersen & Bilfeldt 2016). Busch, Jean-Baptiste, Person, Vaughn Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 12 Method/tool Number of articles (%) 2–3 example articles Detailed example/ description Arts-based (e.g. photovoice, participatory theatre, video, media posters, etc.) 30 (13%) Participatory filmmaking with Qur’anic students in Kano, Nigeria: ‘Speak good about us or keep quiet!’ (Hoechner 2015). Picturing the Wheatbelt: Exploring and expressing place identity through photography (Sonn et al. 2015). Beyond the page: A process review of using ethnodrama to disseminate research findings (Taylor et al. 2017). Based on a participatory docudrama with traditional Qur’anic students in Nigeria, the author reflects on the limitations of participatory research as a tool of empowerment (Hoechner 2015). Interviews, including focus and discussion groups 18 (8%) Everyday mobilisations among grandmothers in South Africa: Survival, support and social change in the era of HIV/AIDS (Chazan 2014). Involving people with intellectual disabilities within research teams: Lessons learned from an Irish experience (García Iriarte et al. 2014). The author conducted interviews and focus groups in South Africa to understand the stresses and strains for ageing women, aka ‘grandmothers’ who are caring for vulnerable children (Chazan 2014). Case study 16 (7%) Governance experiments in water management: From interests to building blocks (Doorn 2016). Enabling organizational cultural change using systemic strategic human resource management – a longitudinal case study (Molineux 2013). The author used a case study design to report on interest groups and stakeholders who developed new water policy (Doorn 2016). Multiple methods/ tools used in same project 15 (6%) Climate-sensitive health priorities in Nunatsiavut, Canada (Harper et al. 2015). Empowering engineering students in ethical risk management: An experimental study (Guntzburger et al. 2019). This study used in-depth interviews, photovoice workshops, and community surveys to examine climate and environmental health priorities in Nunatsiavut, Labrador, Canada (Harper et al. 2015). Table 3 continued Activating social change together: A qualitative synthesis of collaborative change research, evaluation and design literature Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 13 Method/tool Number of articles (%) 2–3 example articles Detailed example/ description Ethnographic- related (e.g. observation, ethnodrama, etc.) 14 (6%) Beyond the page: A process review of using ethnodrama to disseminate research findings (Taylor et al. 2017). ‘We always say: and then came the water…’ Flint’s emergent Latinx capacity building journey during the government-induced lead crisis (Duntley-Matos et al. 2017). Authors describe the process of planning and implementing an ethnodrama to explore men’s health in North Carolina, USA (Taylor et al. 2017). Story/narrative- based 7 (3%) Recipes and revolutions: Consciousness–raising and feminist picnics (Coombs et al. 2016). Creativity as an intervention strategy with Mayan women in Guatemala (Lykes & Crosby 2014). Through participatory creative workshops containing Mayan storytelling and rituals, authors highlight rural Mayan women’s experience and understanding of the 36-year Guatemalan armed conflict (Lykes & Crosby 2014). Design-based 7 (3%) Beyond design and participation: The ‘thought for food’ project in Flanders, Belgium (Cox et al. 2014). Human‐centered design projects and co‐design in/ outside the Turkish classroom: Responses and challenges (Emmanouil 2015). Within a participatory urban design framework, authors used design- based scenarios as the methodological tool for participants to discuss and understand opposing viewpoints about a landscape enhancement project in Belgium (Cox et al. 2014). Reflection 6 (2.5%) Older co-researchers exploring age-friendly communities: An ‘insider’ perspective on the benefits and challenges of peer-research (Buffel 2019). Community health workers as cultural producers in addressing gender-based violence in rural South Africa (de Lange & Mitchell 2016). Older adult co-researchers participated in reflection meetings to share their experiences and viewpoints about the co-research approach used in a study to develop age-friendly communities in the UK (Buffel 2019). Table 3 continued Busch, Jean-Baptiste, Person, Vaughn Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 14 Method/tool Number of articles (%) 2–3 example articles Detailed example/ description Qualitative research method(s) 6 (2.5%) Losing fears and gaining perspective: A case study on youth participation (Aguirre García-Carpintero et al. 2017). Institutional reform and violence reduction in Pernambuco, Brazil (Hoelscher 2017). The purpose of this study was to understand influences of youth participation through the lens of technical workers who work with high school students in the region Els Ports (Castellón). Multiple qualitative methods were used, including interviews, participant observation and group meetings (Aguirre García-Carpintero et al. 2017). CHANGE AND ACTION IN CCRED While some articles implied actions or changes, the literature often does not specify action as a significant aspect of the collaborative change process. Only 14 of the 236 articles (6%) stated a specific action or outcome that occurred as a result of CCRED, and only half of those 14 directly linked the action to a report of change. Eight additional articles reviewed specified a change but did not associate it directly with the action taken. Most often, the creation or modification of a model, method, process, framework or protocol – or the strategies or principles that directed that work—was the change that occurred (n=9; 4%). Modifications to education and training methods and materials were the next most common change indicated in CCRED (n=5; 2%). Quality improvement, policy change and social change (n=8; 3%) exemplified the potentially powerful outcomes that could result from CCRED. Overall, the literature rarely reports change with quantitative or qualitative outcomes. To provide context for the results above, Table 4 provides a concise view of articles we consider exemplars of CCRED. These articles highlight work that displays high levels of collaboration and change-related outcomes while also illustrating a range of terminology, frameworks and methods. Table 4 CCRED exemplars Article Topic Terminology/ Engagement Framework Methods Change/ Outcomes Inclusive research: Making a difference to policy and legislation (Johnson et al. 2014) The article describes two studies and their outcomes to show how people with intellectual disabilities and their supporters can use research that they have done to change policies and laws that affect them. Actively involved in undertaking research Partners in design, content focus and results analysis Empowering research Inclusive qualitative research Qualitative methods: life stories, focus groups playwriting film-making People with disabilities became advocates for change Advocacy led to a change in government policy about sexuality and people with disabilities Table 3 continued Activating social change together: A qualitative synthesis of collaborative change research, evaluation and design literature Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 15 Article Topic Terminology/ Engagement Framework Methods Change/ Outcomes Engaging with communities, engaging with patients: Amendment to the NAPCRG 1998 policy statement on responsible research with communities (Allen et al. 2017) The purpose of this research project was to reconsider a policy statement adopted by the North American Primary Care Research Group (NAPCRG) about responsible participatory research Participatory processes Community responsibility NAPCRG- affiliated research partners Participatory research Workshop Qualitative Analysis NAPCRG endorsed and updated policy statement Young people’s perspectives on participatory ethics: Agency, power and impact in domestic abuse research and policy-making (Houghton 2015) Informed by a larger study about children’s help and solution seeking, this project engaged young expert advisers on domestic abuse policy Young expert advisors Co- developers Participatory Action Research Developed a new, participatory ethical approach that promotes the inclusion and empowerment of young child abuse survivors in research and policy Developing conceptual and methodological foundations in community engagement (Aday, Jr et al. 2015) This article describes the efforts of two undergraduate projects to promote lasting social change for improved health and healthcare in marginalised communities Team members Community partners Social change Individual and collective capacity Community- endorsed Marginalisation Ethnographic descriptions Social network analysis GIS mapping Increased social infrastructure Community- endorsed five- year plan Established regional and international partnerships Brentwood community health care assessment (Goodman et al. 2014) This project assessed health care utilisation and identified existing barriers to health care access Community members Academic- community research partnership Community Based Participatory Research CBPR approaches Survey instrument Improved data collection provided access to a vulnerable community End-of-life conversations and care: An asset-based model for community engagement (Matthiesen et al. 2014) Authors describe use and results of an asset-based approach to facilitate community-led awareness initiatives concerning end-of-life conversations and care Co-creators Involvement Community engagement Community- led awareness Community- identified priorities Asset-based community engagement Four-step process of engagement An approach other communities can use to catalyse community- led awareness initiatives and sustainable community engagement Table 4 continued Busch, Jean-Baptiste, Person, Vaughn Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 16 We observe that all of the above articles begin with a mandate for utilising the results of a participatory process to contribute to social change by producing an actionable outcome. While a further examination of the elements which contribute to successful movement from new knowledge to collective impact/social change is outside the scope of this review, further analysis could contribute to an understanding of how truly effective CCRED can be conducted. Discussion This review sought to document the current understanding, use and measures of CCRED. Taken together, the results of the review suggest that, although various practices are shared across disciplines, CCRED currently operates without a common definition, with limited standards for implementing tools/methods and, most notably, with a lack of standards for evaluating outcomes quality, including whether change and action occurred and their impact. Throughout the review process, we found ourselves frequently asking ‘What counts as collaborative?’ ‘What counts as change?’ and ‘What counts as research?’ We call on all who labour in CCRED arenas to actively work towards a shared understanding of CCRED and from it develop and validate measures to further legitimise it, while continuing to create new methods of approach to CCRED work. Below we consider some important observations and questions regarding CCRED. WHAT TERMINOLOGY IS USED AND HOW IS COLLABORATION DEFINED WITHIN CCRED ACROSS DISCIPLINES? Most reviewed articles (86%) describe ‘participants’ as those from whom data is collected. The terminology here often relies on the naming of some characteristics describing the participants (e.g. age, ethnicity, cultural grouping, place of residence). In contrast, ‘partners or collaborators’, as noted in 14 percent of the articles, participated in the design or implementation of the collaborative change effort. The difference in terminology between participants and partners/ collaborators seems to denote that there is recognition by authors that a greater level of involvement – collaboration – is a distinguishing feature of CCRED and that the nature of the work is changed by this attribute. ‘Community members’ was the most commonly used term for those who collaborate. However, other terminology, such as ‘co-researchers’, ‘co-creators’ and ‘co-enquirers’, was also used and seemed to imply that they were more fully engaged throughout the research process and, at the highest level of engagement, their specific knowledge frameworks were utilised and honoured (Hall & Tandon 2017). We urge those who engage in CCRED efforts to (1) consider that deeper engagement with participants, that is, making an effort to engage them as partners or collaborators, leads to more effective results and greater social change impact, while also being more in line with the emerging ethos of CCRED; and (2) use more specific terminology to define the collaborator roles, in conjunction with ‘levels of engagement’, in order to create a more common understanding across studies. WHAT TYPES OF FRAMEWORKS/APPROACHES ARE BEING USED WITHIN CCRED?  It appears that determining how best to approach CCRED is as complex an undertaking as the problems that the work is attempting to address. While we discovered that some frameworks are more prevalent in education (PAR, AR) and healthcare-related (CBPR) projects, the frameworks and variations identified in our review were used broadly across all Activating social change together: A qualitative synthesis of collaborative change research, evaluation and design literature Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 17 disciplines and industry-based contexts represented within the 236 articles. Additionally, most authors did not utilise specific frameworks exclusively; rather, they drew upon multiple frameworks or approaches to support their CCRED projects. Regarding CCRED frameworks, we question: (1) Do authors lack clarity on distinctions between the various participatory approaches they name? (2) Do authors focus on commonalities rather than distinctions among various frameworks to guide their work? (3) Do CCRED practitioners blend multiple frameworks to create an investigative approach because the available distinct frameworks are inadequate to address context-specific problems? Reliance on multiple frameworks may be, in part, due to the newness and evolving nature of participatory approaches. Burke et al. (2013) provide an example of how participatory research evolves to address context-specific challenges. They describe how Comparative Effectiveness Research (CER) can address newly installed legislative requirements for engaging stakeholders by utilising CBPR practices. Adopting CBPR within CER blends approaches to create new, interdisciplinary investigative frameworks and collaborations. Cope et al. (2019) suggest that a new field of inquiry, Engagement Science, is emerging to help researchers determine which frameworks to apply in order to best engage multiple stakeholders. They describe five traditions of Engaged Science: action research; community-engaged research; citizen science/ public participation in scientific research; team science; and patient-centered research/ patient-centered outcomes research. Their taxonomy is helpful for CCRED project leaders seeking clarity about how participatory frameworks are both connected to and distinct from one another by identifying their origins and their characteristic contributions to participatory research. We suggest that the CCRED community would be well served by a process of collaboratively creating a methodological guidebook which provides definitions, best practices, and guidance around existent and emerging participatory frameworks and methodologies. WHAT ARE THE METHODS AND TOOLS USED IN CCRED? The most commonly mentioned methods/tools described in the reviewed articles were arts- based methods or qualitative methods, such as interviews, case studies and observations. This is not surprising, given that arts-based methods are ideal for CCRED because they allow personal expression and critical inquiry as part of the research process (Thomas & Rappaport 1996) and also have the potential to be supportive, collaborative and participative, and to enhance connections with people so as to share visions for the improvement of society (Lowe 2001). Furthermore, traditional research approaches often do not provide a creative avenue for researchers to express themselves, nor do they increase engagement in the research process (Bringle et al. 2012). The use of interviews and other qualitative research methods serve a purpose in CCRED efforts as they are often more accessible and easily implemented by collaborators not academically trained in research, evaluation or design. Meyer (2000) explains that qualitative methods are frequently used in action research because qualitative research methods facilitate simultaneous focus on the process and the outcomes of change inherent in action research. Almost a third of the reviewed articles did not mention a specific method or tool. It is possible that specific methods and tools were not referred to because authors believed that methods were implied by specifying a participatory approach, such as action research or community-based participatory research. Or they might not have thought it necessary to mention specific methods or tools in the abstracts. One limitation of our work is that we reviewed only abstracts and not the full articles where details on methods or tools might Busch, Jean-Baptiste, Person, Vaughn Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 18 have been provided. Regardless, there is a need for methods and tools that fully engage stakeholders and partners in all aspects of the research process, regardless of their educational or professional background, demographics, or level of experience. To this end, we recommend expanding one’s methodological toolkit for CCRED to become familiar with methods and tools that allow for ease of collaboration and promote transformation/change. For example, Group-Level Assessment (Vaughn & Lohmueller 2014) and concept mapping methodology (Trochim & Kane 2005; Vaughn & McLinden 2016) are two participatory methodologies that actively engage stakeholders in the research process and have action steps built into the process. WHEN CHANGE/ACTION IS DISCUSSED, WHAT IS THE NATURE OF THE CHANGE/ ACTION IN CCRED AND HOW IS IT MEASURED/EVALUATED? In terms of reporting change and action, more questions than answers were provided in the literature we reviewed. Of the 236 abstracts, only 14 clearly indicated that action was taken, while 23 included a report of change. And only half (7) of the articles clearly linked that the action taken was what led to the reporting of change. Numerous questions remain concerning what authors considered ‘action’ and what was reportable ‘change’. As the field of CCRED continues to develop in terms of creating a shared definition and expanding the options for effective methods and tools, it is imperative that we develop measures which define and evaluate the quality of CCRED. Some of the articles in this review reported outcomes such as participants becoming more aware or knowledgeable about an issue. In other articles, the change was in the form of a deliverable – a new protocol, strategy, curriculum, or framework. While such outcomes can be positive contributions to communities, it is difficult to determine if action-oriented change was achieved. We also wrestled with issues of how CCRED is situated in a system that has been created for, and therefore promotes, traditional research. The nature of CCRED is that it is inherently an iterative process and can often be time consuming. It is not unusual for a CCRED practitioner to work with an individual community or on a particular issue for the duration of their career. In an academic system that touts a ‘publish or perish’ mentality, the researcher is forced to write and report ‘change’, no matter its significance or contribution. In our review, we observed that CCRED articles are distributed in publications singularly focused on either process or results. This is a strategy that responds to the demands of the academy and its focus on publication. Many of the articles we reviewed were what we deemed ‘process-oriented’. While there are potentially numerous reasons for this, we suggest it is often due to the centrality of relationship building in conducting CCRED (Silka 2005). We therefore call on CCRED authors to present both process and results as a complete narrative, even when publishing across multiple journals. We suggest, also, the importance of transparently sharing the entirety of work on a project-centric platform (e.g. Research Gate) or, more optimistically, calling for the creation of such a platform specifically for CCRED. Conclusion There is a growing appreciation of the benefits of CCRED, leading to a desire, and increasingly an expectation, that work which seeks to effect positive change be undertaken in collaboration with those for whom the change will be most relevant (e.g. public and patient engagement focus on the Patient-Centered Outcomes Research Institute, patient experience and involvement within England’s National Health Service, etc.). As with any emerging Activating social change together: A qualitative synthesis of collaborative change research, evaluation and design literature Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 19 practice, patterns of productive methodology, as well as gaps which prevent progress, are becoming apparent. It is our hope that this review provides those concerned with practising CCRED an opportunity to reflect on its current state, align their understanding, and begin to move forward in a systematic effort to effectively utilise CCRED to create new knowledge, implement social change, evaluate outcomes, and communicate both failure and success in ways which are accessible and beneficial to all, regardless of role or discipline. References Aday Jr, D, Weeks, J, Sherman, C, Marty, R & Silverstein, R 2019, ‘Developing conceptual and methodological foundations in community engagement’, Journal of Community Engagement and Scholarship, vol. 8, no. 1, p. 3. Aguirre García-Carpintero, A, Moliner Miravet, L & Traver Martí, J 2017, ‘Losing fears and gaining perspective: A case study on youth participation’, Athenea Digital, Revista de pensamiento e investigación social, vol. 17, pp. 237–63. https://doi.org/10.5565/rev/athenea.152 Allen, M, Salsberg, J, Knot, M, Lemaster, J, Felzien, M, Westfall, J, Herbert, C, Vickery, K, Culhane- Pera, K & Ramsden, V 2017, ‘Engaging with communities, engaging with patients: Amendment to the NAPCRG 1998 Policy Statement on Responsible Research with Communities’, Family Practice, vol. 34, pp. 313–21. https://doi.org/10.1093/fampra/cmw074 Andersen, J & Bilfeldt, A 2016, ‘Action research in nursing homes’, Action Research, vol. 14, pp. 19–35. https://doi.org/10.1177/1476750315569082 Ayunerak, P, Alstrom, D, Moses, C, Charlie Sr, J & Rasmus, S 2014, ‘Yup’ik culture and context in southwest Alaska: Community member perspectives of tradition, social change, and prevention’, American Journal of Community Psychology, vol. 54, no. 1, pp. 91–99. https://doi.org/10.1007/s10464-014-9652-4 Balazs, C & Morello-Frosch, R 2013, ‘The three Rs: How community-based participatory research strengthens the rigor, relevance, and reach of science’, Environmental Justice, vol. 6, pp. 9–16. https://doi. org/10.1089/env.2012.0017 Brett, J, Staniszewska, S, Mockford, C, Herron‐Marx, S, Hughes, J, Tysall, C et al. 2014, ‘Mapping the impact of patient and public involvement on health and social care research: A systematic review’, Health Expectations, vol. 17, no. 5, pp. 637–50. https://doi.org/10.1111/j.1369-7625.2012.00795.x Bringle, R, Clayton, P & Price, M 2012, ‘Partnerships in service learning and civic engagement’, Partnerships: A Journal of Service-Learning and Civic Engagement, vol. 1, no. 1. Brown, T & Wyatt, J 2010, ‘Design thinking for social innovation IDEO’, Development Outreach, vol. 12, no. 1, pp. 29–31. https://doi.org/10.1596/1020-797x_12_1_29 Buchanan, D, Miller, F & Wallerstein, N 2007, ‘Ethical issues in community-based participatory research: Balancing rigorous research with community participation in community intervention studies’, Progress in Community Health Partnerships: Research, Education, and Action, vol. 1, no. 2, pp. 153–60. https://doi.org/10.1353/cpr.2007.0006 Buchanan, R 1992, ‘Wicked problems in design thinking’, Design Issues, vol. 8, no. 2, pp. 5–21. https:// doi.org/10.2307/1511637 Buffel, T 2019, ‘Older Coresearchers Exploring Age-friendly Communities: An “insider” perspective on the benefits and challenges of peer-research’, The Gerontologist, vol. 59, no. 3, pp. 538–48. https://doi. org/10.1093/geront/gnx216 Busch, Jean-Baptiste, Person, Vaughn Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 20 https://doi.org/10.5565/rev/athenea.152 https://doi.org/10.1093/fampra/cmw074 https://doi.org/10.1177%2F1476750315569082 https://doi.org/10.1007/s10464-014-9652-4 https://doi.org/10.1089/env.2012.0017 https://doi.org/10.1089/env.2012.0017 https://doi.org/10.1111/j.1369-7625.2012.00795.x https://doi.org/10.1596/1020-797x_12_1_29 https://doi.org/10.1353/cpr.2007.0006 https://doi.org/10.2307/1511637 https://doi.org/10.2307/1511637 https://doi.org/10.1093/geront/gnx216 https://doi.org/10.1093/geront/gnx216 Burke, J, Jones, J, Yonas, M, Guizzetti, L, Virata, M, Costlow, M, Morton, S & Miller, E 2013, ‘PCOR, CER, and CBPR: alphabet soup or complementary fields of health research?’, Clinical and Translational Science, vol. 6, no. 6, pp. 493–96. https://doi.org/10.1111/cts.12064 Cabaj, M & Weaver, L 2016, Collective impact 3.0: An evolving framework for community change, Tamarack Institute, Waterloo, ON, pp. 1–14. Cabassa, L, Camacho, D, Vélez-Grau, C & Stefancic, A 2017, ‘Peer-based health interventions for people with serious mental illness: A systematic literature review, Journal of Psychiatric Research, vol. 84, pp. 80–89. https://doi.org/10.1016/j.jpsychires.2016.09.021 Cargo, M & Mercer, S 2008, ‘The value and challenges of participatory research: Strengthening its practice, Annual Review of Public Health, vol. 29, pp. 325–50. https://doi.org/10.1146/annurev. publhealth.29.091307.083824 Cashman, S, Adeky, S, Allen, A, Corburn, J, Israel, B, Montaño, J et al. 2008, ‘The power and the promise: Working with communities to analyze data, interpret findings, and get to outcomes’, American Journal of Public Health, vol. 98, no. 8, pp. 1407–17. https://doi.org/10.2105/ajph.2007.113571 Chang, C, Minkler, M, Salvatore, A, Lee, P, Gaydos, M & San Liu, S 2013, ‘Studying and addressing urban immigrant restaurant worker health and safety in San Francisco’s Chinatown district: A CBPR case study’, Journal of Urban Health, vol. 90, pp. 1026–40. https://doi.org/10.1007/s11524-013-9804-0 Chazan, M 2014, ‘Everyday mobilisations among grandmothers in South Africa: Survival, support and social change in the era of HIV/AIDS’, Ageing and Society, vol. 34, no. 10, pp. 1641–65. https://doi. org/10.1017/S0144686X13000317 Chen, P, Weiss, F & Nicholson, H 2010, ‘Girls study Girls Inc.: Engaging girls in evaluation through participatory action research, American Journal of Community Psychology, vol. 46, nos 1–2), pp. 228–37. https://doi.org/10.1007/s10464-010-9328-7 Concannon, T, Fuster, M, Saunders, T, Patel, K, Wong, J, Leslie, L & Lau, J 2014, ‘A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research’, Journal of General Internal Medicine, vol. 29, no. 12, pp. 1692–701. https://doi.org/10.1007/s11606-014-2878-x Coombs, C, Franzmann, C, Haynes, R, Holtsclaw, A & Pedersen, C 2016, Recipes and revolutions: Consciousness-raising and Feminist picnics, Feminist Review, vol. 114, no. 1, pp. 130–38. https://doi. org/10.1057/s41305-016-0010-2 Cooper, S 2014, ‘Transformative evaluation: Organisational learning through participative practice’, The Learning Organization, vol. 21, no. 2, pp. 146–57. https://doi.org/10.1108/tlo-03-2013-0003 Cope, E, Angove, R, Dungan, R, Peay, H 2019, ‘Engagement science: An overview of the landscape of engaged research’, Blog: Post Academy Health, www.academyhealth.org/blog/2019-01/engagement- science-overview-landscape-engaged-research Cousins, J & Chouinard, J 2012, ‘Participatory evaluation up close: An integration of research-based knowledge’, IAP, North Carolina, NC. Cousins, J & Whitmore, E 1998, ‘Framing Participatory Evaluation. New Directions for Evaluation, vol. 80, pp. 5-23. Cousins, J, Whitmore, E & Shulha, L 2013, ‘Arguments for a common set of principles for collaborative inquiry in evaluation’, American Journal of Evaluation, vol. 34, no. 1, pp. 7–22. https://doi. org/10.1177/1098214012464037 Activating social change together: A qualitative synthesis of collaborative change research, evaluation and design literature Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 21 https://doi.org/10.1111/cts.12064 https://doi.org/10.1016/j.jpsychires.2016.09.021 https://doi.org/10.1146/annurev.publhealth.29.091307.083824 https://doi.org/10.1146/annurev.publhealth.29.091307.083824 https://doi.org/10.2105/ajph.2007.113571 https://doi.org/10.1007/s11524-013-9804-0 https://doi.org/10.1017/S0144686X13000317 https://doi.org/10.1017/S0144686X13000317 https://doi.org/10.1007/s10464-010-9328-7 https://doi.org/10.1007/s11606-014-2878-x https://doi.org/10.1057/s41305-016-0010-2 https://doi.org/10.1057/s41305-016-0010-2 https://doi.org/10.1108/tlo-03-2013-0003 http://www.academyhealth.org/blog/2019-01/engagement-science-overview-landscape-engaged-research http://www.academyhealth.org/blog/2019-01/engagement-science-overview-landscape-engaged-research https://doi.org/10.1177/1098214012464037 https://doi.org/10.1177/1098214012464037 Cox, V, Goethals, M, De Meulder, B, Schreurs, J & Moulaert, F 2014, ‘Beyond design and participation: The “thought for food” project in Flanders, Belgium’, Journal of Urban Design, vol. 19, no. 4, pp. 412–35. https://doi.org/10.1080/13574809.2014.923742 de Lange, N & Mitchell, C 2016, ‘Community health workers as cultural producers in addressing gender-based violence in rural South Africa’, Global Public Health, vol. 11, nos 5–6, pp. 783–98. https:// doi.org/10.1080/17441692.2016.1170867 Diaz, A, Spears Johnson, C & Arcury, T 2015, ‘Perceptions that influence the maintenance of scientific integrity in community-based participatory research’, Health Education and Behavior, vol. 42, no. 3, pp. 393–401. https://doi.org/10.1177/1090198114560016 Domecq, J, Prutsky, G, Elraiyah, T, Wang, Z, Nabhan, M, Shippee, N, Brito, J, Boehmer, K, Hasan, R, Firwana, B & Erwin, P 2014, ‘Patient engagement in research: A systematic review’, BMC Health Services Research, vol. 14, no. 1, p. 89. Doorn, N 2016, ‘Governance experiments in water management: From interests to building blocks’, Science and Engineering Ethics, vol. 22, no. 3, pp. 755–74. https://doi.org/10.1007/s11948-015-9627-3 Duntley-Matos, R, Arteaga, V, García, A, Arellano, R, Garza, R & Ortega, R 2017, ‘We always say: “and then came the water”: Flint’s emergent latinx capacity building journey during the government-induced lead crisis’, Journal of Community Practice, vol. 25, nos 3–4), pp. 365–90. https://doi.org/10.1080/1070542 2.2017.1384422 Ellis, L & Kass, N 2017, ‘How are PCORI-funded researchers engaging patients in research and what are the ethical implications?’ AJOB Empirical Bioethics, vol. 8, no. 1, pp. 1–10. https://doi.org/10.1080/232 94515.2016.1206045 Emmanouil, M 2015, ‘Human‐centered design projects and co‐design in/outside the Turkish classroom: Responses and challenges’, International Journal of Art and Design Education, vol. 34, no. 3, pp. 358–68. https://doi.org/10.1111/jade.12087 Fetterman, D, Rodríguez-Campos, L, Wandersman, A & O’Sullivan, R 2014, ‘Collaborative, participatory, and empowerment evaluation: Building a strong conceptual foundation for stakeholder involvement approaches to evaluation’ (A response to Cousins, Whitmore & Shulha 2013), American Journal of Evaluation. https://doi.org/10.1177/1098214013509875 Fetterman, D, Rodríguez-Campos L, Zukoski A 2017, Collaborative, participatory, and empowerment evaluation: Stakeholder involvement approaches, Guilford Press, New York. Fetterman, D & Wandersman A (eds) 2005, Empowerment evaluation principles in practice, Guilford Press, New York. Gamoran, 2018, ‘Evidence-based policy in the real world: A cautionary view’, The Annals of the American Academy of Political and Social Science, vol. 678, no. 1, pp. 1180–91. Ganz, M 2010, ‘Leading change: Leadership, organization and social movements’, in N Nohria & R Khurana (eds), Handbook of leadership theory and practice, Harvard Business School Press, Danvers MA, pp. 509–50. García Iriarte, E, O’Brien, P & Chadwick, D 2014, ‘Involving people with intellectual disabilities within research teams: Lessons learned from an Irish experience’, Journal of Policy and Practice in Intellectual Disabilities, vol. 11, no. 2, pp. 149–57. https://doi.org/10.1111/jppi.12081 Busch, Jean-Baptiste, Person, Vaughn Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 22 https://doi.org/10.1080/13574809.2014.923742 https://doi.org/10.1080/17441692.2016.1170867 https://doi.org/10.1080/17441692.2016.1170867 https://doi.org/10.1177/1090198114560016 https://doi.org/10.1007/s11948-015-9627-3 https://doi.org/10.1080/10705422.2017.1384422 https://doi.org/10.1080/10705422.2017.1384422 https://doi.org/10.1080/23294515.2016.1206045 https://doi.org/10.1080/23294515.2016.1206045 https://doi.org/10.1111/jade.12087 https://doi.org/10.1177%2F1098214013509875 https://doi.org/10.1111/jppi.12081 Goodman, M, Gonzalez, M, Gil, S, Si, X, Pashoukos, J, Stafford, J, Ford, E & Pashoukos, D 2014, Brentwood community health care assessment, ‘Progress in community health partnerships’, Research, Education, and Action, vol. 8, no. 1, p. 29. https://doi.org/10.1353/cpr.2014.0017 Gorman, A, 2018, ‘The future of higher education is social impact’, Stanford Social Innovations Review, 18 May 2018. https://ssir.org/articles/entry/the_future_of_higher_education_is_social_impact Guntzburger, Y, Pauchant, T & Tanguy, P 2019, ‘Empowering Engineering students in ethical risk management: An experimental study’, Science and Engineering Ethics, vol. 25, no. 3, pp. 911–37. https:// doi.org/10.1007/s11948-018-0044-2 Hall, B & Tandon, R 2017, ‘Decolonization of knowledge, epistemicide, participatory research and higher education’, Research for All, vol. 1, pp. 6–19. https://doi.org/10.18546/RFA.01.1.02 Harper, S, Edge, V, Ford, J, Willox, A, Wood, M & Mcewen, S 2015, ‘Climate-sensitive health priorities in Nunatsiavut, Canada’, BMC Public Health, vol. 15, no. 1, p. 605. https://doi.org/10.1186/s12889-015- 1874-3 Hayter, C & Cahoy, D 2018, ‘Toward a strategic view of higher education social responsibilities: A dynamic capabilities approach’, Strategic Organization, vol. 16, no. 1, pp. 12–34. https://doi. org/10.1177/1476127016680564 Hoechner, H 2015, ‘Participatory filmmaking with Qur’anic students in Kano, Nigeria: “speak good about us or keep quiet!”’ International Journal of Social Research Methodology, vol. 18, no. 6, pp. 635–49. https://doi.org/10.1080/13645579.2014.929877 Hoelscher, K 2017, ‘Institutional reform and violence reduction in Pernambuco, Brazil’, Journal of Latin American Studies, vol. 49, no. 4, pp. 855–84. https://doi.org/10.1017/s0022216x17000748 Houghton, C 2015, ‘Young people’s perspectives on participatory ethics: Agency, power and impact in domestic abuse research and policy‐making’, Child Abuse Review, vol. 24, no. 4, pp. 235–48. https://doi. org/10.1002/car.2407 Johansson‐Sköldberg, U, Woodilla, J, Çetinkaya, M 2013, ‘Design thinking: Past, present and possible futures’, Creativity and Innovation Management, vol. 22, no. 2, pp. 121–46. https://doi.org/10.1111/ caim.12023 Johnson, K, Minogue, G & Hopklins, R 2014, ‘Inclusive research: Making a difference to policy and legislation’, Journal of Applied Research in Intellectual Disabilities, vol. 27, no. 1, pp. 76–84. https://doi.org/10.1111/jar.12085 Kania, J & Kramer, M 2011, ‘Collective impact’, Stanford Social Innovation Review, vol. 9, no. 1, pp. 36–41. Kidd, S & Kral, M 2005, ‘Practicing participatory action research’, Journal of Counseling Psychology, vol. 52, no. 2, pp. 187–195. http://dx.doi.org/10.1037/0022-0167.52.2.187 Lotz-Sisitka, H, Wals, A, Kronlid, D & McGarry, D 2015, ‘Transformative, transgressive social learning: Rethinking higher education pedagogy in times of systemic global dysfunction’, Current Opinion in Environmental Sustainability, vol. 16, pp. 73–80. https://doi.org/10.1016/j.cosust.2015.07.018 Lowe, S 2001, ‘The art of community transformation’, Education and Urban Society, vol. 33, no. 4, pp. 457–71. https://doi.org/10.1177/0013124501334008 Lykes, M & Crosby, A 2014, ‘Creativity as an intervention strategy with Mayan women in Guatemala’, Intervention, vol. 12, no. 1, pp. 30–42. https://doi.org/10.1097/wtf.0000000000000021 Activating social change together: A qualitative synthesis of collaborative change research, evaluation and design literature Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 23 https://dx.doi.org/10.1353%2Fcpr.2014.0017 https://ssir.org/articles/entry/the_future_of_higher_education_is_social_impact https://doi.org/10.1007/s11948-018-0044-2 https://doi.org/10.1007/s11948-018-0044-2 https://doi.org/10.18546/RFA.01.1.02 https://doi.org/10.1186/s12889-015-1874-3 https://doi.org/10.1186/s12889-015-1874-3 https://doi.org/10.1177/1476127016680564 https://doi.org/10.1177/1476127016680564 https://doi.org/10.1080/13645579.2014.929877 https://doi.org/10.1017/s0022216x17000748 https://doi.org/10.1002/car.2407 https://doi.org/10.1002/car.2407 https://doi.org/10.1111/caim.12023 https://doi.org/10.1111/caim.12023 https://doi.org/10.1111/jar.12085 https://psycnet.apa.org/doi/10.1037/0022-0167.52.2.187 https://doi.org/10.1016/j.cosust.2015.07.018 https://doi.org/10.1177%2F0013124501334008 https://doi.org/10.1097/wtf.0000000000000021 Matthiesen, M, Froggatt, K, Owen, E & Ashton, J 2014, ‘End-of-life conversations and care: An asset- based model for community engagement’, BMJ Supportive and Palliative Care, vol. 4, no. 3, pp. 306–12. http://dx.doi.org/10.1136/bmjspcare-2013-000516 Mertens, D 1999, ‘Inclusive evaluation: Implications of transformative theory for evaluation’, American Journal of Evaluation, vol. 20, no. 1, pp. 1–14. https://doi.org/10.1016/s1098-2140(99)80105-2 Mertens, D 2008, Transformative research and evaluation, Guilford Press, New York. Meyer, J 2000, ‘Using qualitative methods in health related action research’, BMJ, vol. 320, no. 7228, pp. 178–81. doi: https://doi.org/10.1136/bmj.320.7228.178 Molineux, J 2013, ‘Enabling organizational cultural change using systemic strategic human resource management: A longitudinal case study’, The International Journal of Human Resource Management, vol. 24, no. 8, pp. 1588–612. https://doi.org/10.1080/09585192.2012.723022 National Institutes of Health 2011, Principles of community engagement, 2nd edn, Clinical and Translational Science Awards Consortium, Community Engagement Key Function Committee, Task Force on the Principles of Community Engagement, National Institutes of Health, Bethesda, MD. Newman, S, Andrews, J, Magwood, G, Jenkins, C, Cox, M & Williamson, D 2011, ‘Peer reviewed: community advisory boards in community-based participatory research: A synthesis of best processes’, Preventing Chronic Disease, vol. 8, no. 3. Nilsen, E, Myrhaug, H, Johansen, M, Oliver, S & Oxman, A 2006, ‘Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material’, Cochrane Database of Systematic Reviews, 19 July. O’Sullivan, R 2012, ‘Collaborative evaluation within a framework of stakeholder-oriented evaluation approaches’, Evaluation and Program Planning, vol. 35, no. 4, pp. 518–22. https://doi.org/10.1016/j. evalprogplan.2011.12.005 Parker, W & Becker-Benton, A 2016, ‘Experiences in conducting participatory communication research for HIV prevention globally: Translating critical dialog into action through action media’, Frontiers in Public Health, vol. 4, p. 128. https://doi.org/10.3389/fpubh.2016.00128 Patton, M 1997, ‘Toward distinguishing empowerment evaluation and placing it in a larger context’, Evaluation Practice, vol. 18, no. 1, pp. 147–63. https://doi.org/10.1016/s0886-1633(97)90019-2 Person, B, Ali, S, A’Kadir, F, Ali, J, Mohammed, U, Mohammed, K, Rollinson, D & Knopp, S 2016, ‘Community knowledge, perceptions, and practices associated with urogenital schistosomiasis among school-aged children in Zanzibar, United Republic of Tanzania’, PLoS: Neglected Tropical Diseases, vol. 10, no. 7. https://doi.org/10.1371/journal.pntd.0004814 Reason, P & Torbert, W 2001, ‘The action turn: Toward a transformational social science’, Concepts and Transformation, vol. 6, no. 1 , pp. 1–37. https://doi.org/10.1075/cat.6.1.02rea Rosenthal, E, Balcazar, H, De Heer, H, Wise, S, Flores, L & Aguirre, M 2014, ‘Critical reflections on the role of CBPR within a RCT community health worker prevention intervention’, The Journal of Ambulatory Care Management, vol. 37, no. 3, pp. 241–49. https://doi.org/10.1097/jac.0000000000000010 Rossman, G 2000, ‘Participatory monitoring and evaluation’, The Center for International Education, University of Massachusetts, Amherst, MA. http://www.umass.edu/cie/ Themes/participatory_ evaluation.htm Busch, Jean-Baptiste, Person, Vaughn Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 24 http://dx.doi.org/10.1136/bmjspcare-2013-000516 https://doi.org/10.1016/s1098-2140(99)80105-2 https://doi.org/10.1136/bmj.320.7228.178 https://doi.org/10.1080/09585192.2012.723022 https://doi.org/10.1016/j.evalprogplan.2011.12.005 https://doi.org/10.1016/j.evalprogplan.2011.12.005 https://doi.org/10.3389/fpubh.2016.00128 https://doi.org/10.1016/s0886-1633(97)90019-2 https://doi.org/10.1371/journal.pntd.0004814 https://doi.org/10.1075/cat.6.1.02rea https://doi.org/10.1097/jac.0000000000000010 http://www.umass.edu/cie/ Salimi, Y, Shahandeh, K, Malekafzali, H, Loori, N, Kheiltash, A, Jamshidi, E & Majdzadeh, R 2012, ‘Is community-based participatory research (CBPR) useful? A systematic review on papers in a decade’, International Journal of Preventive Medicine, vol. 3, no. 6, p. 386. Shen, S, Doyle‐Thomas, K, Beesley, L, Karmali, A, Williams, L, Tanel, N & McPherson, A 2017, ‘How and why should we engage parents as co‐researchers in health research? A scoping review of current practices, Health Expectations, vol. 20, no. 4, pp. 543–54. https://doi.org/10.1111/hex.12490 Shippee, N, Domecq Garces, J, Prutsky Lopez, G, Wang, Z, Elraiyah, T, Nabhan, M et al. 2015, ‘Patient and service user engagement in research: A systematic review and synthesized framework’, Health Expectations, vol. 18, no. 5, pp. 1151–66. https://doi.org/10.1111/hex.12090 Silka, L 2005, Reconfiguring applied research: Research partnerships as opportunities for innovation, Monograph, Scholarship in Action. Sonn, C, Quayle, A & Kasat, P 2015, ‘Picturing the wheatbelt: Exploring and expressing place identity through photography’, American Journal of Community Psychology, vol. 55, nos 1–2), pp. 89–101. https:// doi.org/10.1007/s10464-014-9686-7 Sullivan, L, Klein, E, Brown, T, Sample, M, Pham, M, Tubig, P, Folland, R, Truitt, A & Goering, S 2018, ‘Keeping disability in mind: A case study in implantable brain–computer interface research’, Science and Engineering Ethics, vol. 24, no. 2, pp. 479–504. https://doi.org/10.1007/s11948-017-9928-9 Taylor, J, Namey, E, Carrington Johnson, A & Guest, G 2017, ‘Beyond the page: A process review of using ethnodrama to disseminate research findings’, Journal of Health Communication, vol. 22, no. 6, pp. 532–44. https://doi.org/10.1080/10810730.2017.1317303 Tegegne, A, Penker, M & Wurzinger, M 2016, ‘Participatory demographic scenarios addressing uncertainty and transformative change in Ethiopia’, Systemic Practice and Action Research, vol. 29, no. 3, pp. 277–96. https://doi.org/10.1007/s11213-015-9365-0 Thomas, J & Harden, A 2008, ‘Methods for the thematic synthesis of qualitative research in systematic reviews’, BMC Medical Research Methodology, vol. 8, no. 1, p. 45. https://doi.org/10.1186/1471-2288-8-45 Thomas, R & Rappaport, J 1996, ‘Art as community narrative: A resource for social change’, in M Lykes, ed., Myths about the powerless: Contesting social inequalities, Temple University Press, Philadelphia, PA, pp. 317–36. Trochim, W & Kane, M 2005, ‘Concept mapping: An introduction to structured conceptualization in health care’, International Journal for Quality in Health Care, vol. 17, no. 3, pp. 187–91. https://doi. org/10.1093/intqhc/mzi038 van Winkelen, C 2016, ‘Using developmental evaluation methods with communities of practice’, The Learning Organization, vol. 23, nos 2/3, pp. 141–55. https://doi.org/10.1108/TLO-08-2015-0047 Vaughn, L, Jacquez, F, Zhen-Duan, J, Graham, C, Marschner, D, Peralta, J, García, H, Recino, M, Maya, M, Maya, E & Cabrera, M 2017, ‘Latinos Unidos por la Salud: The process of developing an immigrant community research team’, Collaborations: A Journal of Community-Based Research and Practice, vol. 1, no. 1, p. 2. https://doi.org/10.33596/coll.9 Vaughn, L & Lohmueller, M 2014, ‘Calling all stakeholders: Group-level assessment – A qualitative and participatory method for large groups’, Evaluation Review, vol. 38, 4, pp. 336–55. https://doi. org/10.1177/0193841X14544903 Vaughn, L & McLinden, D 2016, ‘Concept mapping’, in L Jason, D Glenwick & P Firm, eds, Handbook of methodological approaches to community-based research: qualitative, quantitative, and Activating social change together: A qualitative synthesis of collaborative change research, evaluation and design literature Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 25 https://doi.org/10.1111/hex.12490 https://doi.org/10.1111/hex.12090 https://doi.org/10.1007/s10464-014-9686-7 https://doi.org/10.1007/s10464-014-9686-7 https://doi.org/10.1007/s11948-017-9928-9 https://doi.org/10.1080/10810730.2017.1317303 https://doi.org/10.1007/s11213-015-9365-0 https://doi.org/10.1186/1471-2288-8-45 https://doi.org/10.1093/intqhc/mzi038 https://doi.org/10.1108/TLO-08-2015-0047 https://doi.org/10.33596/coll.9 https://doi.org/10.1177%2F0193841X14544903 https://doi.org/10.1177%2F0193841X14544903 mixed methods, Oxford University Press, New York, pp. 305–14. https://doi.org/10.1093/med:psy ch/9780190243654.003.0030 Wallerstein, N, Duran, B, Oetzel, J & Minkler, M (eds) 2017, Community-based participatory research for health: Advancing social and health equity, Jossey-Bass, San Francisco, CA. Weisbord, M 2012, ‘Productive workplaces: Dignity, meaning, and community in the 21st century’, John Wiley & Sons, Hoboken, NJ. Zukoski, A & Luluquisen, M 2002, ‘Participatory evaluation. What is it? Why do it? What are the Challenges?’, Community-Based Public Health Policy and Practice: Partnership for the Public’s Health, Vol. 5, pp. 1–6. Busch, Jean-Baptiste, Person, Vaughn Gateways: International Journal of Community Research and Engagement, Vol. 12, No. 2, December 2019 26 https://doi.org/10.1093/med:psych/9780190243654.003.0030 https://doi.org/10.1093/med:psych/9780190243654.003.0030