Gateways: International Journal of Community Research and Engagement Vol. 15, No. 1 June 2022 © 2022 by the author(s). This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International (CC BY 4.0) License (https:// creativecommons.org/licenses/ by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercial, provided the original work is properly cited and states its license. Citation: Plummer, R., Witkowski, S., Smits, A., and Dale, G. 2022. Appraising HEI-community Partnerships: Assessing Performance, Monitoring Progress, and Evaluating Impacts. Gateways: International Journal of Community Research and Engagement, 15:1, 1–17. https://doi.org/10.5130/ijcre. v15i1.8014 ISSN 1836-3393 | Published by UTS ePRESS | http://ijcre. epress.lib.uts.edu.au RESEARCH ARTICLE (PEER-REVIEWED) Appraising HEI-community Partnerships: Assessing Performance, Monitoring Progress, and Evaluating Impacts Ryan Plummer1, Samantha Witkowski1, Amanda Smits1, Gillian Dale1 1 Environmental Sustainability Research Centre, Brock University, St. Catharines, Ontario, Canada, L2S 3A1 Corresponding author: Ryan Plummer; rplummer@brocku.ca DOI: https://doi.org/10.5130/ijcre.v15i1.8014 Article History: Received 11/12/2021; Revised 13/04/2022; Accepted 24/05/2022; Published 06/2022 Abstract Momentum of the creation of partnerships between higher education institutions (HEIs) and communities is strong. As their significance intensifies, the question of how to judge their value is garnering increasing attention. In this perspective article, we develop a framework for comprehensively appraising HEI-community partnerships. Constituent parts of the framework are unpacked, and application of the framework is then discussed. The appraisal framework provides a mechanism to document evidence of worth, and most importantly contributes to the continuous improvement and learning imperative of HEI- community partnerships. Keywords Appraisal; Continuous Improvement; Evaluation; HEI-Community Partnerships; Monitoring; Performance Assessment Introduction The relationship between higher education institutions (HEIs) and their community is being reimagined amidst pressing global challenges, the changing nature of knowledge production, and dialogue regarding the functions of higher education in contemporary society ( Jongbloed et al. 2008; Nelson 2021). Sentiment is widespread that HEIs should meaningfully contribute 1 DECLARATION OF CONFLICTING INTEREST The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. FUNDING The author(s) received no financial support for the research, authorship, and/or publication of this article. https://creativecommons.org/licenses/ by/4.0/ https://creativecommons.org/licenses/ by/4.0/ https://creativecommons.org/licenses/ by/4.0/ https://doi.org/10.5130/ijcre.v15i1.8014 https://doi.org/10.5130/ijcre.v15i1.8014 http://ijcre.epress.lib.uts.edu.au http://ijcre.epress.lib.uts.edu.au mailto:rplummer@brocku.ca https://doi.org/10.5130/ijcre.v15i1.7832 to society (e.g. Compagnucci & Spigarelli 2020; Hart & Northmore 2010). This outlook has captured the attention of governments, funders and non-governmental organisations (de Boer et al. 2015; Plummer et al. 2021a; Secundo et al. 2017). Over the past two decades, the emphasis on strengthening relationships with communities has increased (e.g. Bawa & Munck 2012; Tremblay 2017) and community engagement itself is being recognised as central to the mandate of HEIs (Nelson 2021; Plummer et al. 2021a; Secundo et al. 2017). HEI-community relationships take many forms and encompass the triumvirate of research, teaching and service. It is heuristic to conceive of these forms along a spectrum of participation, with consultation being the least interactive and collaboration being the most (IAP2 2014). We focus on partnerships between HEIs and community because they specifically help HEIs meet engagement mandates, as well as enable alignment of HEI functions with community needs (Groulx et al. 2021). Similarly to many terms that have gained prominence in popular lexicon, the term partnership in this context is imprecisely employed and variously understood (Drahota et al. 2016; Luger et al. 2020). We discuss this challenge in the section that follows, concentrating on formal partnerships because they are a prominent means for HEIs and the community to actively engage with diverse societal challenges and to realise opportunities; offer explicit and agreed upon parameters regarding goal(s), functioning and aspirations; afford context specificity as determined in the initiation phase by the partners themselves (Estrella & Gaventa 1998), and have been recognised as an appropriate unit of analysis, which is an intervening variable as well as an ‘… outcome of “impact” in itself ’ (Cruz & Giles 2000, p. 31). The myriad of potential benefits from HEI-community partnerships is increasingly being recognised (Buys & Bursnall 2007; Groulx et al. 2021; Holton et al. 2015; Meza et al. 2016; Williamson et al. 2016). Such collaborations leverage access to knowledge and expertise, thereby facilitating diagnosis of community challenges as well as formulation of potential solutions (Buys & Bursnall 2007; Hart & Northmore 2010; Meza et al. 2016; Williamson et al. 2016). Engaging community partners through these types of partnerships aspires to eliminate tokenism and connect content experts (professionals, staff within each organisation, service providers, etc.) with context experts (those with lived experience (Attygalle 2017). Additionally, capacity-building opportunities for community partners increase (Holliday et al. 2015; Holton et al. 2015; Muse 2018; Williamson et al. 2016), leading to more resilient communities. At the same time, such collaboration broadens the range of perspectives considered and diversity of knowledge employed (Meza et al. 2016). It also ideally enables learning that is reciprocal (Holliday et al. 2015). Ultimately, partnerships serve to advance the exchange of knowledge and education, enhance community practice and inform decision-making (Buys & Bursnall 2007; Holton et al. 2015; Muse 2018; Wlliamson et al. 2016). As the importance of HEI-community relationships increases and excitement about potential benefits from partnerships grows, the need to gauge success is intensifying. This concerted concern for measuring success stems from the accountability agenda in higher education (Holton et al. 2015; Shephard 2018; Wiek et al. 2012). Despite these intensifying needs, ‘there is a lack of consensus in the field regarding what defines partnership success …’ (Brush et al. 2019, p. 1). As HEI-community partnerships continue to rise in prominence and importance, there is a clear need to devise corresponding mechanisms to identify, gauge and track the success of these unique arrangements (Holton et al. 2015; Nelson 2021; Plummer et al. 2021a). In this article, we examine how to gauge the success of formal HEI-community partnerships. As a perspective article, it is essential at the outset to detail the authors’ positionality. The authors are all affiliated with the Environmental Sustainability Research Centre (ESRC) at Brock University in Canada. Sustainability science is an overarching imperative for the Centre, which aims ‘… to strengthen the exchange and integration of different disciplinary and non-academic knowledge, enabling mutual learning between scientists and practitioners’ (Brandt et al. 2013, pp. 1–2). Innovative partnerships were initiated by the ESRC in the spirit of bridging the science–societal divide and as such encompass our research, education and service endeavours. Partnerships are established with both government and non- Plummer, et al. Gateways: International Journal of Community Research and Engagement, Vol. 15, No. 1 June 20222 governmental organisations, with Memoranda of Understanding (MOUs) setting out the aims, associated responsibilities and budgetary contributions of all parties. The importance of communities was elevated at Brock University in the present strategic plan for the institution (Brock University 2018) and advanced in the associated Community Engagement Strategic Plan (Brock University 2020, p. 1): the ‘… connection to community is fundamental to the University’s strategic mission. Community engagement will support each of Brock’s strategic objectives’. In seeking to draw upon current experience, the President encouraged the authors to catalyse community connections across the institution as well as determine how to gauge the success of formal community partnerships. The authors thus embarked on a multi-year program of inquiry into the performance of higher education institution– community partnerships (Plummer et al. 2021a), including measuring the performance of sustainability science initiatives (Plummer et al. 2022) and evaluating transdisciplinary partnerships for sustainability (Plummer et al. 2021b). While sustainability science informed some of the empirical investigations, the program of inquiry transcended disciplinary, geographic and institutional contexts. With the experience and perspective of the authors acknowledged, we start by scrutinising challenges that make this a particularly vexing dilemma – terminological and conceptual differences, multiple perspectives and ways of knowing, inconsistency about what constitutes success, and measurement messiness. We then provide a conceptual framework for measuring the success of a partnership, from both HEI and community partner perspectives, that is clear, cohesive and comprehensive. The manner by which the framework can be operationalised within HEI-community partnerships is then detailed. The proposed framework bolsters the capacity of HEI-community partnerships to navigate key challenges and gauge their success. Measuring the Success of HEI-community Partnerships: A Quagmire Individuals interested in measuring the success of HEI-community partnerships are immediately confronted with a paradox. On the one hand, there are numerous accounts appraising HEI-community partnerships in great detail (e.g. Brinkerhoff 2002; Butterworth & Palermo 2008; Holton et al. 2015; McNall et al. 2009; Plummer et al. 2021b; Tyndall et al. 2020; Wiek et al. 2012; Williamson et al. 2016). On the other hand, consensus does not exist between scholars as to what constitutes HEI-community partnership success (Brush et al. 2019). Often, studies focused on appraisal (e.g. Buys & Bursnall 2007; Holton et al. 2015; Wiek et al. 2012) address success from a HEI perspective and the view of the community partner(s) is absent (Hart & Northmore 2010). Additionally, many existing accounts and frameworks focus only on a single component of HEI-community partnerships. For example, Blackstock et al. (2007) provide a framework for undertaking a summative evaluation of participatory research. Similarly, Van Tulder (2016) provides an analytical framework for partnership impact assessments. While these frameworks offer a snapshot of the evaluation component, which takes place near the end of the partnership, they do not adequately capture every component throughout the duration of the partnership process, including how they are interconnected and shape HEI-community partnerships throughout their lifecycle. Contrastingly, Jagosh et al. (2015) undertook a realist evaluation of 11 concluded research-based partnerships in order to understand the contextual factors (e.g. trust, power sharing, resource sharing, etc.) that impacted the overall outcomes of the partnership. In this example, the appraisal is more invested in the partnership process and what circumstances led to various outcomes. Ultimately, these accounts and proposed frameworks do not present a holistic view of partnerships or encompass all essential components and considerations of HEI-community partnerships. Scholars are explicit regarding the requirement to better understand and appraise HEI-community partnerships (Drahota et al. 2016; Holton et al. 2015; Luger et al. 2020; Nelson 2021). These calls are complemented by arguments regarding the need for mechanisms to identify, gauge and track success Plummer, et al. Gateways: International Journal of Community Research and Engagement, Vol. 15, No. 1 June 20223 (Holton et al. 2015; Plummer et al. 2021a; Van Tulder et al. 2016). In the following section we address key challenges which contribute to this contradiction and create an entangled dilemma in measuring the success of HEI-community partnerships. TERMINOLOGICAL AND CONCEPTUAL DIFFERENCES Terminological and conceptual differences pose a substantive challenge. The term partnership itself is imprecisely used and variously understood (Drahota et al. 2015; Luger et al. 2020; Nelson 2021; Plummer et al. 2021a). Drahota et al.’s (2015) systematic review on this matter found that terms used to describe partnerships ranged broadly, from community-academic partnerships and community-based participatory research (CBPR) partnerships to university-community partnerships, community-university partnerships and academic-community partnerships. This may be explicable in the light of varying degrees of formalisation (Brinkerhoff 2002) and/or discipline-specific terminology (Horton et al. 2009). In this article, we follow the comprehensive definition by Brinkerhoff (2002, p. 21), who describes a partnership as a ‘… dynamic relationship among diverse actors, based on mutually agreed objectives, pursued through a shared understanding of the most rational division of labour based on the respective comparative advantages of each partner. Partnership encompasses mutual influence, with a careful balance between synergy and respective autonomy, which incorporates mutual respect, equal participation in decision-making, mutual accountability and transparency.’ As Plummer et al. (2021a, p. 2) observe, ‘one immediate challenge within this area of literature is the imprecise and interchangeable uses of terms such as “relationship”, “engagement”, and “partnership”’. These terms are often colloquially used to describe partnerships when they actually describe very different circumstances. This is further complicated by the tendency for disciplines and communities of practice to also define partnership in different ways, leading to misunderstandings and inconsistencies across disciplinary boundaries and fields of practice (Horton et al. 2009). There are also more fundamental conceptual differences in HEI-community partnerships. Drahota et al. (2015) highlight the extent of these disparities in their systematic review. For example, CBPR partnerships are understood there as arrangements among structurally unequal groups that come together to address problems such as poverty, crime and housing (Drahota et al. 2015). However, Benoit et al. (2005, p. 265) describe partnership arrangements as ‘a process of ongoing negotiation through which academic and community partners establish their respective expectations and responsibilities in the partnership, always taking into account changes in personnel, agendas, and budget allocations, among other things’. The conceptualisation of ‘community-academic partnership’ (CAP) is used by Drahota et al. (2015) to reconcile understanding across different disciplines and the need for equitable control. MULTIPLE PERSPECTIVES AND DIFFERENT WAYS OF KNOWING  A considerable challenge of HEI-community partnerships is ‘respecting, balancing, bridging, reconciling and/or sometimes integrating differing knowledge systems, values and processes among disciplines and with partner communities, which translate into potentially differing assumptions about what constitutes effective interaction and credible knowledge generation’ (Steelman et al. 2021, p. 633). HEI-community partnerships are evident across the gamut of scholarly areas and, depending on the discipline, they can take different forms and encompass varying activities (Ansari et al. 2001; Plummer et al. 2021a). The involvement of individuals from different professions, disciplines and scholarly areas brings unique challenges with regard to integrating their different perspectives (Blecher & Hughes 2020). The magnitude of this challenge is amplified by the emphasis on transdisciplinarity (Brandt et al. 2013; Lang et al. 2012), combined with an emphasis on community partnerships. Plummer, et al. Gateways: International Journal of Community Research and Engagement, Vol. 15, No. 1 June 20224 Scholarship on measuring partnerships has largely emphasised the view of HEIs (e.g. Buys & Bursnall 2007; Sargent & Waters 2004; Wiek et al. 2012) and often excludes the community perspective (Sandy & Holland 2006). Hart and Northmore (2010, p. 6) observe that ‘the rigorous and comprehensive incorporation of community perspectives in audit and benchmarking is almost entirely absent across the higher education sector ...’. However, the importance of the perspective of community partners is critical (Srinivas et al. 2015) as it provides a more complete picture of how the partnership is operating and/ or the overall success of the partnership (Plummer et al. 2021a). This situation is complicated as HEI and community partners may enter into a partnership for different reasons, have divergent expectations and uniquely perceive benefits, all of which affect gauging success (Klein 2008; Plummer et al. 2021a; Sandy & Holland 2006). Moreover, cogent areas, professions and communities may have implicitly and/ or explicitly established methods and associated criteria for evaluating partnership success (Belcher et al. 2016). For example, research evaluation approaches may emphasise measures of academic outputs, such as publication of books or prestige journals, whereas community organisations may emphasise the number of people attending a public partnership event or the number of responses to a community-based survey. Also, evaluation criteria from more than one discipline or community of practice may be contradictory or conflicting (Gaziulusoy et al. 2016; Klein 2006). HEI-community partnerships pose a particular challenge in this regard as they are often context-specific (Hansson & Polk 2018). Ultimately, ‘the failure to grapple with understanding the community perspective may have potentially dire consequences because there is considerable room for misunderstandings between higher education and community partners …’ (Sandy & Holland 2006, p. 31). It is imperative to be cognisant of the multiple perspectives that may be underpinned by unique ways of knowing and manifest knowledge systems (Plummer et al. 2022; Steelman et al. 2021) and that actors must accommodate, reconcile and/or integrate different knowledges (Norström et al. 2020). This poses challenges as each can be viewed as legitimate, valid and credible. For example, Indigenous methods of evaluation may be very different from those of Western academics. Western approaches tend to focus on objectivity and the systematic collection and analysis of data, whereas an Indigenous method of evaluation may be a decolonising and spiritual process of deep reflection (Evans et al. 2020). Further, it is important to acknowledge that certain ways of knowing have become entrenched, so HEIs may be reluctant to create space for different ways of knowing for fear that they may be perceived as less credible or legitimate. WHAT CONSTITUTES ‘SUCCESS’? Measuring the success of HEI-community partnerships requires shared understanding, while defining success is difficult as there are multiple perspectives. Linquist-Grantz and Vaughn (2016) focus on the relationship between process and outcomes, whereby individual contexts within the processes of a partnership are critically examined to rank the success of the community-academic partnership. Ensuring not only multiple perspectives are included within the process, but also individual contexts are considered within the partnership evaluation framework is considered critical. As with the scholarship on measuring partnerships, often studies (e.g. Buys & Bursnall 2007; Holton et al. 2015; Wiek et al. 2012) lack inclusion of the perspectives of the community partner(s) (Hart & Northmore 2010). Much of the literature exploring what constitutes partnership success resides in the field of community- based participatory research (e.g. Brush et al. 2019; Israel et al. 2020; Luger et al. 2020). Brush et al. (2019, p. 565) identified 28 indicators of success and concluded that the multi-dimensional construct goes beyond outcomes and includes ‘some combination of characteristics of partners, relationships among/between partners, partnership characteristics, processes, resources and capacity, along with partnership outcomes’. Not only are indicators of success numerous, but much like indicators for evaluation, they tend to be discipline specific, with many disciplines having well-established criteria and processes for evaluation. Plummer, et al. Gateways: International Journal of Community Research and Engagement, Vol. 15, No. 1 June 20225 Beyond the nature of the data itself, HEIs are ill-equipped to collect the required data to track community partnership work and, more specifically, measure the success of such partnerships as most existing enterprise systems focus on the institution’s instruction and academic aspects (Holton et al. 2015). A MEASUREMENT MESS  A final challenge is the actual measurement of success. Measurement nomenclature is often used interchangeably and employed imprecisely (see Plummer et al. 2021b). Each has a specific meaning and purpose, which we describe in the next section. There are also broad monikers for subject areas or fields, and developments within these further exacerbate confusion (Ansari et al. 2001). For example, evaluation is a field of study with numerous specific types therein (e.g. formative evaluation, program evaluation, impact evaluation, process evaluation, etc.), which causes blurring of boundaries and the potential to collect or focus on imprecise and/or irrelevant information in relation to the HEI-community partnership.   Issues in nomenclature are compounded by operational and conceptual measurement considerations. Many measurement tools and case studies exist, but these come from diverse disciplines/fields (e.g. Benoit et al. 2005 in public health/medicine, and Weik et al. 2021 in sustainability science; see Drahota et al. 2016 and Luger et al. 2020 for others). However, they tend to be singularly focused and measure different aspects of HEI-community partnerships. For example, Buys and Bursnall (2007) focus on indicators for understanding the partnership process, without making connections to outcomes or impacts. Contrastingly, Azaroff et al. (2011) primarily concentrate on evaluating the impacts of a public health campaign on the choices of community members. Despite the existence of numerous measurement tools, ‘… empirical evaluation of [HEI-community partnerships] are inadequate’ (Drahota et al. 2016, p. 195). The foregoing leads Luger et al. (2020, p. 509) to observe that while ‘many practical lessons learned, and conceptual models can be found in the literature ... models and concepts of engaged research still remain muddy’. A Proposed Framework for Capturing HEI-community Partnership Success While the aforementioned challenges are substantial and difficult, we argue that appraising success is imperative to strengthen HEI-community partnerships as well as address matters of accountability, transparency and value. We endeavour to overcome these challenges and advance this subject by (1) providing clear definitions and parameters of assessment, monitoring and evaluation; and (2) conceptually establishing a comprehensive framework for the consistent measurement of HEI-community partnership inputs, processes, outcomes and impacts.   Our framework (Figure 1) builds upon foundational work on assessing the performance of HEI- community partnerships by Plummer et al. (2020). It sets out and illustrates the relationship among three salient components for appraising HEI-community partnerships: assessment, monitoring and evaluation. Each component is briefly summarised and then the overall workings of the framework are addressed. At the outset, it is imperative to recognise that the three components are dynamic and not mutually exclusive; they connect in complementary ways throughout the appraisal process to measure the success of a given HEI-community partnership. Component 1: Assessment Assessment involves identifying the present status of a given partnership in reference to desired conditions and determining how to close gaps in performance. This includes ‘comparing the current condition to the desired condition, defining the problem or problems, understanding the behaviours and mechanisms that contribute to the current conditions, determining if and how specific behaviors and mechanisms can be Plummer, et al. Gateways: International Journal of Community Research and Engagement, Vol. 15, No. 1 June 20226 changed to produce the desired condition, developing solution strategies, and building support for action’ (Gupta et al. 2007, pp. 14–15). It occurs through the systematic, continuous collection of data on specified indicators to provide insight into the current performance of the partnership itself (Caplan et al. 2007; Estrella & Gaventa 1998; Onyango 2018; Stem et al. 2005). Assessment takes place after the initiation/ formalisation of a partnership through a memorandum of understanding (MOU), and can occur iteratively throughout the lifecycle of the partnership, as illustrated in Figure 1. Qualities associated with the performance of partnerships are abundant. In drawing upon the recent synthesis of this scholarship in relation to HEI-community partnerships (Plummer et al. 2020), we highlight the qualities that comprise a good partnership: the circumstances surrounding their formation (inputs), how they function (processes), and what they may accomplish (outcomes). These three categories have recently been established and their associated qualities have been validated as appropriate indicators and measures for assessing partnership performance (Plummer et al. 2021a). We provide a synopsis of each below (see also Plummer et al. 2021a for the full suite of associated qualities and indicators). INPUTS Inputs are the contributions bestowed to the enterprise by the entities at the initiation of the partnership (Figure 1). Effective communication during initiation is imperative as each entity articulates their motivation for seeking entry into formal partnership and thereby a commitment to the jointly shared aims. Clarity at the outset on what each entity will contribute to the partnership is also important. These supports should align with the aims of the partnership as they will shape the process as well as the capacity for what can be accomplished. The availability of financial resources, for example, is often identified as a key influence on many other aspects of a partnership (e.g. scope, duration, activities, etc. (Buys & Bursnall 2007; Holton et al. 2015; Sargent & Waters 2004). Resources can also dictate the roles of particular collaborators (Sargent Figure 1. A conceptual framework to appraise success of HEI-community partnerships (adapted from Plummer et al. 2020) Plummer, et al. Gateways: International Journal of Community Research and Engagement, Vol. 15, No. 1 June 20227 & Waters 2004). Human resources (citizens, experts, faculty, students, staff ) clearly play an important role in success, so developing networks is highlighted as cultivating connections among actors may enhance opportunities for different types and sources of support (Bringle & Hatcher 2002; Holton et al. 2015; Sargent & Waters 2004; Schulz et al. 2003). PROCESS The second category in the assessment portion of the framework (Figure 1) is process (Plummer et al. 2020). Process refers to how the partnership functions (McNall et al. 2009). A process which promotes respectful and constructive interactions between/among actors is imperative for success as the partnership transitions from initiation to implementation (Sargent & Waters 2004). It is important that the process fosters mutual understanding of perspectives, a collective understanding of the problem, and a basis for joint decision-making (Amey & Brown 2005; McNall et al. 2009; Schulz et al. 2003). Effective communication, established in the initiation phase, continues to be imperative for smooth operation of a given partnership (Amey & Brown 2005; Bringle & Hatcher 2002; Holton et al. 2015; Schulz et al. 2003). The process also engenders the essential quality of trust, so that individuals come to expect no harm from the others involved, and ideally develop confidence that others will complete actions if their control is relinquished (Mayer et al. 1995). Reciprocity and mutual respect, which often develops as trust is built, are further qualities of successful partnerships. It is especially important to recognise in the context of HEI-community partnerships that individuals bring unique perspectives, skills and contributions to the process, and it is this diversity that strengthens the partnership process (Schulz et al. 2003). OUTCOMES The final category of outcomes in the assessment portion of the framework (Figure 1) encompasses what is produced as well as the effects on those directly involved in the partnership (Plummer et al. 2020). The realisation of outcomes is broadly influenced by the employment of available resources and the effectiveness of the process in goal attainment (Schulz et al. 2003). Outcomes are not temporally constrained and can occur throughout the life of the partnership (Koontz & Thomas 2012). The three-fold typology of outcomes for partnerships (Plummer et al. 2020; Sargent & Waters 2004) highlights objective outcomes, subjective outcomes and learning outcomes. Objective outcomes are tangible products (e.g. publications, reports, etc.) that are easily quantifiable and often used as measures of productivity in reporting. Subjective outcomes (e.g. satisfaction, trust, etc.) have value for, and are interpreted by, those involved in the partnership. Finally, learning outcomes include knowledge creation and acquisition, integration of diverse perspectives to address multi-party challenges, and skill development and attainment (Amey & Brown 2005; Sargent & Waters 2004). Component 2: Monitoring Monitoring involves the practice of conducting systematic observations to gain information about progress (Estrella & Gaventa 1998; Onyango 2018). Data is collected using indicators that provide insight into aspects of the partnership as well as activities (Stem et al. 2005). Monitoring should take place regularly over the life cycle of the HEI-community partnership, as illustrated in Figure 1. It provides routine feedback, and thereby a basis for continuous improvement (Estrella & Gaventa 1998) and learning. Although the importance of monitoring partnerships is recognised within the literature (Calderon & Mathies 2013; Kagan & Duggan 2009), HEIs acknowledge that the ability to track and measure partnerships is a key challenge in practice (Plummer et al. 2021a). Secundo et al. (2017, p. 232) observe that ‘the nature of relevant data required to track third mission activities is considered as invisible, tacit, Plummer, et al. Gateways: International Journal of Community Research and Engagement, Vol. 15, No. 1 June 20228 unquantifiable, informal, and in most cases, it is not collected by administrators’ (see also Shephard 2018; 2020). Reflective of, and contributing to, the challenges experienced in practice, there is currently no consensus within the literature regarding monitoring procedures/documentation or indicators of success (Kagan & Duggan 2009), or exactly how to institutionalise tracking across disciplines/types of partnerships. Monitoring of HEI-community partnerships is two-fold. First, it involves tracking performance over time and determining progress (Buys & Bursnall 2007; Pellegrino et al. 2014), while assessment considers a single point in time. Second, monitoring focuses on activities that take place throughout the partnership (Bäckstrand 2006) and, specifically, on the achievement of activities as milestones, signalling advancement toward their aims, goals and/or objectives. Monitoring is typically accomplished through tracking variables, or key performance indicators (KPIs). KPIs are measurable indicators used to signal progress or achievements against pre-defined standards or objectives. Traditionally, and especially in an academic context, KPIs are often quantitative and used in order to obtain ‘objective’ data for evaluation purposes (Garlick & Langworthy 2008). However, as partnerships require both objective and subjective outcomes, it is important that KPIs for appraisal represent a wide range of qualitative (e.g. perceptions) and quantitative (e.g. number of academic publications) information, which may be tracked for different purposes (Garlick & Langworthy 2008; Plummer et al. 2022). In line with the two-fold purpose of monitoring above, KPIs for HEI-community partnerships come from existing scholarship as well as the specific context of a given partnership. Monitoring HEI-community partnership performance over time, for example, can benefit greatly from adopting and systematically applying indicator measures which are established and validated in relation to HEI-community partnership performance for each category described above (Plummer et al. 2021a), and are also transferable to different types of partnerships (disciplines, stages, etc.). Conversely, KPIs regarding actions or activities that signal progress towards their aim, goals and/or objectives are unique, and therefore need to be established and agreed upon by the partners. Component 3: Evaluation Evaluation is a field of inquiry in its own right and comprises diverse conceptualisations (Ansari et al. 2001; Shephard 2020). Generally, evaluation entails determining the extent to which objectives have been achieved and intended impacts realised (Onyango 2018). It is prudent for evaluation to occur near the end of the partnership (see Figure 1) for three reasons. First, the duration of a partnership is typically identified in a formal agreement and should align with expectations that the objectives can be realistically accomplished over this time. Second, information gathered through monitoring should enable ongoing adjustments to keep the partnership on track and inform evaluation. Third, evaluation near the conclusion of a partnership serves as a mechanism to consider what has been achieved and how to move forward. The partners may wish to continue their collaboration, revise their formal agreement and the accompanying aims, or cease working together. As evaluation of HEI-community partnerships is essential, a ‘culture of evidence’ has emerged whereby HEIs and institutions more broadly must increasingly document their performance, achievements and impacts (Calderon & Mathies 2013). This culture has largely been driven by the issue of accountability – the ‘obligation to report to others, to explain, to justify, to answer questions about how resources have been used and to what effect’ (Trow 1996, p. 310). Emergence of the accountability agenda is widespread (de Boer et al. 2015; Jongebloed 2018), being embraced and influential in shaping public administration in a business- like way (Huisman & Currie 2004; Plummer et al. 2021a). Although clearly an essential process, evaluation of HEI-community partnerships is, surprisingly, an uncommon practice. For example, a recent study of Canadian HEIs found that most HEIs ‘occasionally’ use some form of monitoring and evaluation, and 25 percent of institutions reported that they do not employ Plummer, et al. Gateways: International Journal of Community Research and Engagement, Vol. 15, No. 1 June 20229 any form of monitoring or evaluation (Plummer et al. 2021a). Evaluative studies of HEI-community partnerships are also rare. In one notable example, Bowen and Martens (2006) tested a collaborative ‘utilisation focused’ approach to evaluation, whereby community partners were meaningfully incorporated into planning, developing and evaluating all aspects of the partnership/project. More recently, Shephard (2018) investigated how HEI-community partnerships were valued at the University of Otago and showed that partnerships used different types of evaluation, including explicit and formalised evaluations, informal evaluations and implicit evaluations. Efforts in both practice and scholarship are hampered by the lack of guidelines and consensus on how to effectively evaluate HEI partnerships and their impacts. This often leads to the imposition of ineffective, inadequate and varied approaches to describe, understand and evaluate both their activities and impacts (Shephard 2020). This problem is exacerbated by the contextual specificity of both evaluation and partnerships. In an effort to address the above matters, evaluation of the proposed framework specifically focuses on effectiveness and impact. Effectiveness evaluation measures whether the objectives of the partnership have been achieved (Bowen & Martens 2006; Deniston et al. 1968). As illustrated in Figure 1, the agreed upon objectives are typically codified in a document (e.g. an MOU) and are an expression of the desired outcomes of the partnership. They imply one or more necessary conditions which must be met to result in accomplishment (Deniston et al. 1968). As such, the effectiveness indicators are context specific and determined in the initiation phase by the partners themselves (Estrella & Gaventa 1998). Relatedly, impact evaluation is concerned with the causal effects from an intervention (such as a partnership). It encompasses ‘positive and negative, primary and secondary long-term effects, directly or indirectly, intended or unintended’ (OECD-DAC 2010, p. 24). It is important to give consideration to the temporal dimension of impact assessment in this context. While some of the intended impacts from the partnership are set out during the initiation phase, it is important that the partners are open to unforeseen impacts and document these throughout the partnership life cycle. Additionally, impacts may develop after the partnership has officially ended (as determined in the MOU). It then becomes more difficult to isolate long-term effects or impacts, both logistically and resource wise (Van Tulder et al. 2016). Impacts are distinguished from outcome assessment and effectiveness evaluation in two ways. First, impacts refer to the extent to which the intervention (partnership) has made a societal difference (Van Tulder et al. 2016; Worton et al. 2017). Societal audiences, as illustrated in Figure 1, may include students, governments, non-government organisations, industry, academics, citizens, and so on. Second, impacts need to be measured from the perspective of these different audiences (Srinivas et al. 2015). A guide to assessing performance and evaluating the impacts of transdisciplinary partnerships for sustainability by Plummer et al. (2022) outlines key steps in the evaluation process and provides a navigational pathway forward through the process. While the techniques to carry out effectiveness and impact evaluation will be unique to each HEI-community partnership, the suggested framework can assist with overcoming challenges, offer mechanisms to document evidence of achievements and ultimately provide a foundation for transformative change. Practical Application of the Framework While undoubtedly some HEI-community partnerships implicitly incorporate one or more of the components set out above, to the best of our knowledge there is currently no appraisal framework which brings together assessment, monitoring and evaluation. The scholarly basis for such an appraisal framework is provided above. However, the appraisal framework has little heuristic value if it cannot be applied in practice. Here we consider application of the appraisal framework in relation to frequently observed life-cycle stages of HEI-community partnerships (cf Bringle & Hatcher 2002; Lewinson 2014). Although the Plummer, et al. Gateways: International Journal of Community Research and Engagement, Vol. 15, No. 1 June 202210 timeframe in which HEI-community partnerships go through these stages varies, we illustrate them unfolding over a five-year formal partnership. Our intent is to provide a general appraisal guide applicable to HEI-community partnerships, while also acknowledging the specific parts that are context dependent and ultimately require the sound judgement of the entities involved in the appraisal. An appropriate entry point is creation of a HEI-community partnership. The reason(s) for collaboration is paramount at this initiation stage. Instrumental rationale for partnering includes gaining particular knowledge, leveraging complementary skills, gaining access to unique opportunities, and so on. Potential enjoyment from collaboration and cultivating enduring relationships is a common intrinsic motivation. These motivations often precipitate formal codification through a MOU, which sets out the aims, duration, scope, funding contributions, legal considerations, and so on. Ideally, the partners consider when/how often assessment, monitoring and evaluation will occur as well as what each will entail. The performance assessment component of the appraisal framework coincides with the implementation phase of the partnership lifecycle. In this phase, entities are concerned with enacting collaboration. Assessment gauges performance at a particular point in time. There is no prescribed timeline for conducting performance assessment; however, it is important that assessment is continuous so as to provide ongoing feedback for the duration of the partnership (Figure 1). From our experience with HEI-community partnerships, we suggest that performance assessment is undertaken annually as this allows sufficient time for tangible consideration while also permitting ideal response time for feedback. The HEI-Community Partnership Performance Index (HCPPI; Plummer et al. 2021a) is a recent validated rapid assessment tool that can easily be completed by HEI and community participants to assess the performance of their collaboration. It involves a 47-item questionnaire designed to provide a comprehensive understanding of a given HEI-community partnership’s strengths and areas for improvement across the three broad categories of assessment (inputs, process, and outcomes). As partnerships make incremental adjustments based on this evidence, they employ an adaptive approach for continuous improvement (Folke et al. 2005; Lee 1994). This adaptive approach aligns well with the dynamism and complexity of HEI-community partnerships (Plummer et al. 2020). Monitoring coincides with the implementation phase of the partnership lifecycle and is considered iteratively throughout (Figure 1). It captures progress of an HEI-community partnership in terms of performance and progress towards its aims, goals and objectives. In regard to the former, monitoring provides information on how the partnership is performing over time. In applying the appraisal framework, this part of monitoring is a logical extension of performance assessment and is achieved by tracking the results of the HCPPI over time. More specifically, the results obtained within the first year provide a baseline or benchmark for subsequent performance. Monitoring performance thus affords a barometer by which the partners can gauge their maintenance, improvement and/or deterioration in particular aspects. Including all entities in the partnership is critical to accurately gauge performance as well as learn from feedback and make continuous improvements. Monitoring also concentrates on determining progress. These milestones should be established at the outset of the partnership and revisited as necessary. Determining KPIs that capture the essence of the overall goals or milestones should occur at the outset of the partnership. These KPIs manifest as indicators to measure progress towards goals. KPIs can be qualitative and/or quantitative, and may adapt or change over time as the partnership evolves. A given milestone or goal may have multiple relevant KPIs. The most important consideration regarding KPIs is that they reflect the overall goals and are agreed upon by all entities. Monitoring progress may occur at different times in different partnerships, depending on the length of the partnership and the intended milestones. Often, multi-year partnerships identify yearly milestones and deliverables. Plummer, et al. Gateways: International Journal of Community Research and Engagement, Vol. 15, No. 1 June 202211 Evaluation of the framework offers guides to practically tackle critical questions about HEI-community partnership success, including effectiveness and impact. The intentions of the partnership are typically articulated at the outset in some type of document (Figure 1) or are developed in the formative stages of the partnership lifecycle. When setting out intentions, partnerships will benefit greatly from articulating what they aspire to accomplish as well as whom they seek to affect. In so doing, the stated aims offer a clear understanding of the desired effects and impacts. Specificity in these intentional statements (or sub- statements) is strongly encouraged to facilitate the operationalisation of evaluation. While intentions are mainly established at the outset, partners need to be aware of, and open to, emergent opportunities in terms of accomplishments as well as audiences. Evaluation typically occurs at the completion stage of the partnership life cycle (Sargent & Waters 2004) and entails two main considerations. Evaluating effectiveness is anchored to the goals and objectives established at the outset of the partnership, and is made operational by agreed upon indicators or milestones. Whereas monitoring documents’ progress to evaluate effectiveness uses KPIs and other credible evidence to cumulatively judge achievements. Critically, all entities in the partnership should participate in evaluating effectiveness, thereby coming to a shared understanding of accomplishments. Impacts are a related consideration of evaluation when applying the framework. Aspirational societal impacts as a consequence of the partnership are commonly discussed during the initiation phase and evolve over the lifespan of the project. Identifying specific target audiences associated with the intended societal impacts, cultivating relationships with those audiences, and devising ways to assess if they have experienced changes due to the partnership are all essential. Engaging multiple, and often diverse, audiences in evaluation can be logistically difficult, expensive and time-consuming. There are, however, numerous quantitative and qualitative tools that are well suited to conducting impact evaluation. Questionnaires, interviews and/or workshops are powerful examples of means to gain insight into partnership impacts from the perspective of multiple and diverse audiences. Conclusion HEI-community partnerships are on a steady upward trajectory. Appraising their success is of increasing importance; however, a myriad of challenges immediately confronts those interested in this enterprise. These include terminological and conceptual differences, multiple perspectives and ways of knowing, inconsistency about what constitutes success, and measurement messiness. In addition to confronting this quagmire, a rigorous and dynamic framework for appraising HEI-community partnerships does not yet exist (Holton et al. 2015; Plummer et al. 2021a; Srinivas et al. 2015). In this perspective article, we set out a conceptual basis for a comprehensive appraisal framework which encompasses the complementary components of assessment, monitoring and evaluation. The framework provides a general guide to transferability across HEI-community partnerships. Assessment considers the present state in relation to aspects and qualities. Monitoring provides important information regarding how the partnership is performing, as well as the impacts created through the partnership. Evaluation provides evidence of goal attainment and demonstrates the societal benefits of a partnership. In discussing how to apply the framework in practice, we have taken an initial step in reconciling conceptual and applied considerations when appraising HEI-community partnerships. Employing the appraisal framework in a variety of HEI-community partnerships is needed next. Flexibility and adaptation are encouraged as consideration of context as well as the perspectives of the participants is paramount. As experience using the framework accumulates, opportunities will emerge for lesson learning, transferability and further refinement. Plummer, et al. Gateways: International Journal of Community Research and Engagement, Vol. 15, No. 1 June 202212 Acknowledgements The authors would like to express their appreciation to the two anonymous reviewers for their feedback. We are also extremely grateful to the Managing Editor and Editorial Team at Gateways for their constructive encouragement and positive support, leading to a much stronger manuscript. Finally, we thank Brock University for championing and supporting community partnerships. References Amey, M & Brown, D 2005, ‘Interdisciplinary collaboration and academic work: A case study of a university- community partnership’, New Directions for Teaching and Learning, vol. 102, pp. 23–35. https://doi.org/10.1002/tl.194 Ansari, W, Ceri, J & Hammick, M 2001, ‘Collaboration and partnerships: Developing the evidence base’, Health and Social Care in the Community, vol. 9, no. 4, pp. 215–27. https://doi.org/10.1046/j.0966-0410.2001.00299.x Attygalle, L 2017, The context experts, viewed 22 November 2021, www.tamarackcommunity.ca/library/the-context-experts Azaroff, L, Nguyen, H, Do, T, Gore, R & Goldstein-Gelb, M 2011, ‘Results of a Community-University Partnership to reduce deadly hazards in hardwood floor finishing’, Journal of Community Health, vol. 36, pp. 658–68. https://doi. org/10.1007/s10900-011-9357-7 Bäckstrand, K 2006, ‘Multi-stakeholder partnerships for sustainable development: Rethinking legitimacy, accountability, and effectiveness’, European Environment, vol. 16, pp. 290– 306. https://doi.org/10.1002/eet.425 Bawa, A & Munck, R 2012, ‘Foreword: Globalizing civic engagement’, in R Munck, Higher education and civic engagement: Comparative perspectives, Palgrave MacMillan, New York, p. xixix. Belcher, B & Hughes, K 2020, ‘Understanding and evaluating the impact of integrated problem-oriented research programmes: Concepts and considerations’, Research Evaluation, vol. 20, no. 2, pp. 154–68. https://doi.org/10.1093/ reseval/rvaa024 Belcher, B, Rasmussen, K, Kemshaw, M & Zornes D 2016, ‘Defining and assessing research quality in a transdisciplinary context’, Research Evaluation, vol. 25, pp. 1–17. https://doi.org/10.1093/reseval/rvv025 Benoit, C, Jansson, M, Millar, A & Phillips, R 2005, ‘Community-academic research on hard-to-reach populations: Benefits and challenges’, Qualitative Health Research, vol. 15, no. 2, pp. 263–82. https://doi. org/10.1177/1049732304267752 Blackstock, K, Kelly, G & Horsey, B 2007, ‘Developing and applying a framework to evaluate participatory research for sustainability’, Ecological Economics, vol. 60, no. 4, pp. 726–42. https://doi.org/10.1016/j.ecolecon.2006.05.014 Bowen, S & Martens, P 2006, ‘A model for collaborative evaluation of university-community partnerships’, Journal of Epidemiology and Community Health, vol. 60, no. 10, pp. 902–907. https://doi.org/10.1136/jech.2005.040881 Brandt, P, Ernst, A, Gralla, F, Luederitz, C, Lang, D, Newig, J, Reinert, F, Abson, D & von Wehrden, H 2013, ‘A review of transdisciplinary research in sustainability science’, Ecological Economics, vol. 92, pp. 1–15. https://doi.org/10.1016/j. ecolecon.2013.04.008 Bringle, R & Hatcher, J 2002, ‘Campus-community partnerships: the terms of engagement’, Journal of Social Issues, vol. 58, no. 3, pp. 503–16. https://doi.org/10.1111/1540-4560.00273 Brinkerhoff, J 2002, ‘Assessing and improving partnership relationships and outcomes: A proposed framework’, Evaluation and Program Planning, vol. 25, pp. 215–31. https://doi.org/10.1016/S0149-7189(02)00017-4 Brock University 2018, Niagara roots–Global reach: Brock University Institutional Strategic Plan 2018–2025, viewed 4 May 2022. https://brocku.ca/strategic-plan/ Plummer, et al. Gateways: International Journal of Community Research and Engagement, Vol. 15, No. 1 June 202213 https://doi.org/10.1002/tl.194 https://doi.org/10.1046/j.0966-0410.2001.00299.x http://www.tamarackcommunity.ca/library/the-context-experts https://doi.org/10.1007/s10900-011-9357-7 https://doi.org/10.1007/s10900-011-9357-7 https://doi.org/10.1002/eet.425 https://doi.org/10.1093/reseval/rvaa024 https://doi.org/10.1093/reseval/rvaa024 https://doi.org/10.1093/reseval/rvv025 https://doi.org/10.1177/1049732304267752 https://doi.org/10.1177/1049732304267752 https://doi.org/10.1016/j.ecolecon.2006.05.014 https://doi.org/10.1136/jech.2005.040881 https://doi.org/10.1016/j.ecolecon.2013.04.008 https://doi.org/10.1016/j.ecolecon.2013.04.008 https://doi.org/10.1111/1540-4560.00273 https://doi.org/10.1016/S0149-7189(02)00017-4 https://brocku.ca/strategic-plan/ Brock University 2020, Brock University’s Community engagement strategic plan: Regional community builder, viewed 4 May 2022. https://brocku.ca/community-engagement/documents/ Brush, B, Mentz, G, Jensen, M, Jacobs, B, Saylor, K, Rowe, Z, Israel, B & Lachance, L 2019, ‘Success in long-standing community-based participatory research (CBPR) partnerships: A scoping literature review’, Heath Education & Behavior, vol. 47, no. 4, pp. 1–13. https://doi.org/10.1177/1090198119882989 Butterworth, I & Palermo, J 2008, ‘Indicators for effective partnerships: Organisational implications for measuring service to international, national and local communities’, Australasian Journal of University Community Engagement, vol. 3, no. 2, pp. 16–27. Buys, N & Bursnall, S 2007, ‘Establishing university-community partnerships: Processes and benefits’, Journal of Higher Education Policy and Management, vol. 29, no. 1, pp. 73–86. https://doi.org/10.1080/13600800601175797 Calderon, A & Mathies, C 2013, ‘Institutional research in the future: Challenges within higher education and the need for excellence in professional practice’, in A Calderon & K Webber (eds), Global issues in institutional research: New directions for institutional research, Jossey Bass, San Francisco, pp. 77–90. https://doi.org/10.1002/ir.20040 Caplan, K, Gomme, J, Mugabi, J & Stott, L 2007, Assessing partnership performance: Understanding the drivers for success. Building partnerships for development in water and sanitation (BPD), viewed 10 December 2020. https://www. joinforwater.ngo/es/assessing-partnership-performance Compagnucci, L & Spigarelli, F 2020, ‘The Third Mission of the university: A systematic literature review on potentials and constraints’, Technological Forecasting and Social Change, vol. 161, 120284. https://doi.org/10.1016/j. techfore.2020.120284 Cruz, N & Giles, D 2000, ‘Where’s the community in service-learning research?’, Michigan Journal of Community Service Learning, pp. 28–34. de Boer, H, Jongbloed, B, Benneworth, P, Cremonini, L, Kolster, R, Kottmann, A, Lemmens-Krug, K & Vossensteyn, H 2015, ‘Performance-based funding and performance agreements in fourteen higher education systems’, viewed 1 July 2019. https://ris.utwente.nl/ws/portalfiles/portal/5139542/jongbloed+ea+performance-based-funding-and- performance-agreements-in-fourteen-higher-education-systems.pdf Deniston, O, Rosenstock, I & Getting, V 1968, ‘Evaluation of program effectiveness’, Public Health Reports, vol. 83, no. 4, pp. 323–35. https://doi.org/10.2307/4593279 Drahota, A, Meza, R, Brikho, B, Naaf, M, Estabillo, J, Gomez, E, Vejnoska, S, Dufek, S, Stahmer, A & Aarons G 2016, ‘Community-Academic Partnerships: A systematic review of the state of the literature and recommendations for future research’, The Milbank Quarterly, vol. 94, no. 1, pp. 163–214. https://doi.org/10.1111/1468-0009.12184 Estrella, M & Gaventa, J 1998, Who counts reality? Participatory monitoring and evaluation: A Literature Review (70), IDS, Brighton, viewed 1 July 2019. https://www.ids.ac.uk/publications/who-counts-reality-participatory-monitoring- and-evaluation-a-literature-review/ Evans, J, Bremmer, L, Johnston, A, Rowe, G & Sasakamoose, J 2020, ‘Exploring Indigenous approaches to evaluation and research in the context of victim services and support’, viewed 19 November 2021. https://publications.gc.ca/ collections/collection_2021/jus/J2-495-2020-eng.pdf Folke, C, Hahn, T & Olsson, P 2005, ‘Adaptive governance of social-ecological systems’, Annual Review of Environment and Resources, vol. 30, pp. 441–73. https://doi.org/10.1146/annurev.energy.30.050504.144511 Garlick, S & Langworthy, A 2008, ‘Benchmarking university community engagement: Developing a national approach in Australia’, Higher Education Management and Policy, vol. 20, no. 2, pp. 105–17. https://doi.org/10.1787/hemp-v20- art17-en Plummer, et al. Gateways: International Journal of Community Research and Engagement, Vol. 15, No. 1 June 202214 https://brocku.ca/community-engagement/documents/ https://doi.org/10.1177/1090198119882989 https://doi.org/10.1080/13600800601175797 https://doi.org/10.1002/ir.20040 https://www.joinforwater.ngo/es/assessing-partnership-performance https://www.joinforwater.ngo/es/assessing-partnership-performance https://doi.org/10.1016/j.techfore.2020.120284 https://doi.org/10.1016/j.techfore.2020.120284 https://ris.utwente.nl/ws/portalfiles/portal/5139542/jongbloed+ea+performance-based-funding-and-performance-agreements-in-fourteen-higher-education-systems.pdf https://ris.utwente.nl/ws/portalfiles/portal/5139542/jongbloed+ea+performance-based-funding-and-performance-agreements-in-fourteen-higher-education-systems.pdf https://doi.org/10.2307/4593279 https://doi.org/10.1111/1468-0009.12184 https://www.ids.ac.uk/publications/who-counts-reality-participatory-monitoring-and-evaluation-a-literature-review/ https://www.ids.ac.uk/publications/who-counts-reality-participatory-monitoring-and-evaluation-a-literature-review/ https://publications.gc.ca/collections/collection_2021/jus/J2-495-2020-eng.pdf https://publications.gc.ca/collections/collection_2021/jus/J2-495-2020-eng.pdf https://doi.org/10.1146/annurev.energy.30.050504.144511 https://doi.org/10.1787/hemp-v20-art17-en https://doi.org/10.1787/hemp-v20-art17-en Gaziulusoy, A, Ryan, C, McGrail, S, Chandler, P & Twomey, P 2016, ‘Identifying and addressing challenges faced by transdisciplinary research teams in climate change research’, Journal of Cleaner Production, vol. 123, pp. 55–64. https:// doi.org/10.1016/j.jclepro.2015.08.049 Groulx, M, Nowak, N, Levy, K & Booth, A 2021, ‘Community needs and interests in university-community partnerships for sustainable development’, International Journal of Sustainability in Higher Education, vol. 22, no. 2, pp. 247–90. https://doi.org/10.1108/IJSHE-03-2020-0086 Gupta, K, Sleezer, C & Russ-Eft, D 2007, A practical guide to needs assessment, 2nd edn, Pfeiffer, San Francisco. Hansson, S & Polk, M 2018, ‘Assessing the impact of transdisciplinary research: The usefulness of relevance, credibility, and legitimacy for understanding the link between process and impact’, Research Evaluation, vol. 27, no. 2, pp. 132–44. https://doi.org/10.1093/reseval/rvy004 Hart, A & Northmore, S 2010, ‘Auditing and evaluating university-community engagement: Lessons from a UK case study’, Higher Education Quarterly, vol. 65, no. 1, pp. 34–58. https://doi.org/10.1111/j.1468-2273.2010.00466.x Holliday, M, DeFalco, T & Sherman, J 2015, ‘Putting impact first: Community-University Partnerships to advance authentic neighborhood sustainability’, Metropolitan Universities, vol. 26, no. 3, pp. 79–104. Holton, V, Early, J, Jettner, J & Shaw, K 2015, ‘Measuring community-university partnerships across a complex research university: Lessons and findings from a pilot enterprise data collection mechanism’, Metropolitan Universities, vol. 26, no. 2, pp. 99–124. Horton, D, Prain, G & Thiele, G 2009, Perspectives on partnerships: A literature review, International Potato Centre (CIP), Lima. Huisman, J & Currie, J 2004, ‘Accountability in higher education: Bridge over troubled water?’, Higher Education, vol. 48, no. 4, pp. 529–51. https://doi.org/10.1023/B:HIGH.0000046725.16936.4c IAP2 2014, Public Participation Spectrum, viewed 22 November 2021, https://www.iap2.org.au/wp-content/ uploads/2019/07/IAP2_Public_Participation_Spectrum.pdf Israel, B, Lachance, L, Coombe, C, Lee, S, Jensen, M, Wilson-Powers, E, Mentz, G, Muhammad, M, Rowe, Z, Reyes, A & Brush, B 2020, ‘Measurement approaches to partnership success: Theory and methods for measuring success in long- standing Community-Based Participatory Research Partnerships’, Progress in Community Health Partnerships: Research, Education, and Action, vol. 14, no. 1, pp. 129–40. https://doi.org/10.1353/cpr.2020.0015 Jagosh, J, Bush, P, Salsberg, J, Macaulay, A, Greenhalgh, T, Wong, G, Cargo, M, Green, L, Herbert, C & Pluye, P 2015, ‘A realist evaluation of community-based participatory research: Partnership synergy, trust building and related ripple effects’, BMC Public Health, vol. 15, pp. 1–11. https://doi.org/10.1186/s12889-015-1949-1 Jongbloed, B, Enders, J & Salerno, C 2008, ‘Higher education and its communities: Interconnections, interdependencies and a research agenda’, Higher Education, vol. 56, pp. 303–24. https://doi.org/10.1007/s10734-008-9128-2 Kagan, C & Duggan, K 2009, ‘Breaking down barriers: Universities and communities working together’, viewed 16 December 2020. http://e-space.mmu.ac.uk/83457/1/978-1-900139-29-8.pdf Klein, J 2006, ‘Afterword: The emergent literature on interdisciplinary and transdisciplinary research evaluation’, Research Evaluation, vol. 15, pp. 75–80. https://doi.org/10.3152/147154406781776011 Klein, J 2008, ‘Evaluation of interdisciplinary and transdisciplinary research’, American Journal of Preventative Medicine, vol. 35, no. 2, pp. 116–23. https://doi.org/10.1016/j.amepre.2008.05.010 Koontz, T & Thomas, C 2012, ‘Measuring the performance of public–private partnerships: Asystematic method for distinguishing outputs from outcomes’, Public Performance & Management Review, vol. 35, no. 4, pp. 769–86. https://doi. org/10.2753/PMR1530-9576350410 Plummer, et al. Gateways: International Journal of Community Research and Engagement, Vol. 15, No. 1 June 202215 https://doi.org/10.1016/j.jclepro.2015.08.049 https://doi.org/10.1016/j.jclepro.2015.08.049 https://doi.org/10.1108/IJSHE-03-2020-0086 https://doi.org/10.1093/reseval/rvy004 https://doi.org/10.1111/j.1468-2273.2010.00466.x https://doi.org/10.1023/B:HIGH.0000046725.16936.4c https://www.iap2.org.au/wp-content/uploads/2019/07/IAP2_Public_Participation_Spectrum.pdf https://www.iap2.org.au/wp-content/uploads/2019/07/IAP2_Public_Participation_Spectrum.pdf https://doi.org/10.1353/cpr.2020.0015 https://doi.org/10.1186/s12889-015-1949-1 https://doi.org/10.1007/s10734-008-9128-2 http://e-space.mmu.ac.uk/83457/1/978-1-900139-29-8.pdf https://doi.org/10.3152/147154406781776011 https://doi.org/10.1016/j.amepre.2008.05.010 https://doi.org/10.2753/PMR1530-9576350410 https://doi.org/10.2753/PMR1530-9576350410 Lang, D, Wiek, A, Bergmann, M, Stauffacher, M, Martens, P, Moll, P, Swilling, M & Thomas, C 2012, ‘Transdisciplinary research in sustainability science: Practice, principles, and challenges’, Sustainability Science, vol. 7, pp. 25–43. https://doi.org/10.1007/s11625-011-0149-x Lee, K 1994, Compass and gyroscope: Integrating science and politics for the environment, Island Press, Washington, US. Lewinson, T 2014, ‘Building a university-community partnership to explore health challenges among residents at extended-stay hotels’, Metropolitan Universities, vol. 25, no. 2, pp. 89–106. Lindquist-Grantz, R & Vaughn, L 2016, ‘The journey and destination need to be intentional: Perceptions of success in Community-Academic Research Partnerships’, Gateways: International Journal of Community Research and Engagement, vol. 9, no. 1, pp. 1–21. https://doi.org/10.5130/ijcre.v9i1.4872 Luger, T, Hamilton, A & True, G 2020, ‘Measuring Community-Engaged Research contexts, process and outcomes: A mapping review’, The Milbank Quarterly, vol. 98, no. 2, pp. 493–53. https://doi.org/10.1111/1468-0009.12458 Mayer, R, Davis, J & Schoorman, F 1995, ‘An integrative model of organizational trust’, Academy of Management Review, vol. 20, pp. 709–34. https://doi.org/10.2307/258792 McNall, M, Reed, C, Brown, R & Allen, A 2009, ‘Brokering community–university engagement’, Innovative Higher Education, vol. 33, no. 5, pp. 317–31. https://doi.org/10.1007/s10755-008-9086-8 Meza, R, Drahota, A & Spurgeon, E 2016, ‘Community-Academic partnership participation’, Community Mental Health Journal, vol. 52, no. 7, pp. 793–98. https://doi.org/10.1007/s10597-015-9890-4 Muse, S 2018, Exploring the community impact of Community-University partnerships, Electronic theses and dissertations, University of Denver, viewed 22 November 2021. https://digitalcommons.du.edu/etd/1415 Nelson, R 2021, ‘Developing evaluable principles for Community–University Partnerships’, Journal of Community Engagement and Scholarship, vol. 14, no. 1, pp. 1–19. https://doi.org/10.54656/JDGK9182 Norström, A, Cvitanovic, C, Löf, M, West S, Wyborn C, Balvanera P, Bednarek, A, Bennett, E et al. 2020, ‘Principles for knowledge co-production in sustainability research’, Nature Sustainability, vol. 3, pp. 182–90. https://doi.org/10.1038/ s41893-019-0448-2 OECD-DAC 2010, Glossary of key terms in evaluation and results-based management, Paris, viewed 14 December 2020. www.oecd.org/development/peer-reviews/2754804.pdf Onyango, R 2018, ‘Participatory monitoring and evaluation: An overview of guiding pedagogical principles and implications on development’, International Journal of Novel Research in Humanity and Social Sciences, vol. 5, no. 4, pp. 428–33. Pellegrino, J, Wilson, M, Koenig, J & Beatty, A (eds) 2014, Developing assessments for the next generation science standards, National Academies Press, Washington, DC. Plummer, R, Blythe, B, Gurney, G, Witkowski S & Armitage, D 2022, ‘Transdisciplinary partnerships for sustainability: An evaluation guide’, Sustainability Science, vol. 17, pp. 955–67. https://doi.org/10.1007/s11625-021-01074-y Plummer, R, Witkowski, S, Smits, A & Dale, G 2020, ‘Assessing the performance of higher education institution (HEI)–community partnerships’, unpublished manuscript. Plummer, R, Witkowski, S, Smits, A & Dale G 2021a, ‘The issue of performance in higher education institution– community partnerships: A Canadian perspective’, Journal of Higher Education Policy and Management, vol. 43, no. 5, pp. 537–56. https://doi.org/10.1080/1360080X.2020.1858386 Plummer, R, Witkowski, S, Smits, A & Dale, G 2021b, ‘Higher education institution–community partnerships: Measuring the performance of sustainability science initiatives’, Innovative Higher Education, vol. 47, pp. 135–53. https://doi.org/10.1007/s10755-021-09572-8 Plummer, et al. Gateways: International Journal of Community Research and Engagement, Vol. 15, No. 1 June 202216 https://doi.org/10.1007/s11625-011-0149-x https://doi.org/10.5130/ijcre.v9i1.4872 https://doi.org/10.1111/1468-0009.12458 https://doi.org/10.2307/258792 https://doi.org/10.1007/s10755-008-9086-8 https://doi.org/10.1007/s10597-015-9890-4 https://digitalcommons.du.edu/etd/1415 https://doi.org/10.54656/JDGK9182 https://doi.org/10.1038/s41893-019-0448-2 https://doi.org/10.1038/s41893-019-0448-2 http://www.oecd.org/development/peer-reviews/2754804.pdf https://doi.org/10.1007/s11625-021-01074-y https://doi.org/10.1080/1360080X.2020.1858386 https://doi.org/10.1007/s10755-021-09572-8 Sandy, M & Holland, B 2006, ‘Different worlds and common ground: Community partner perspectives on campus– community partnerships’, Michigan Journal of Community Service Learning, vol. 13, no. 1, pp. 30–43. Sargent, L & Waters, L 2004, ‘Careers and academic research collaborations: An inductive process framework for understanding successful collaborations’, Journal of Vocational Behavior, vol. 64, pp. 308–19. https://doi.org/10.1016/j. jvb.2002.11.001 Schulz, A, Israel, B & Lantz, P 2003, ‘Instrument for evaluating dimensions of group dynamics within community- based participatory research partnerships’, Evaluation and Program Planning, vol. 26, pp. 249–62. https://doi. org/10.1016/S0149-7189(03)00029-6 Secundo, G, Perez, S, Martinaitis, Ž & Leitner, K 2017, ‘An Intellectual Capital framework to measure universities’ third mission activities’, Technological Forecasting and Social Change, vol. 123, pp. 229–39. https://doi.org/10.1016/j. techfore.2016.12.013 Shephard, K 2018, ‘Valuing and evaluating community-engaged scholarship’, Tertiary Education and Management, vol. 24, pp. 83–94. https://doi.org/10.1080/13583883.2017.1395904 Shephard, K 2020, Higher Education for sustainability: Seeking intellectual independence in Aotearoa New Zealand, Springer, Singapore. https://doi.org/10.1007/978-981-15-1940-6 Srinivas, T, Meenan, C, Drogin, E & DePrince A 2015, ‘Development of the community impact scale measuring community organization perceptions of partnership benefits and costs’, Michigan Journal of Community Service Learning, vol. 21, no. 2, pp. 5–21. Steelman, T, Bogdan, A, Mantyka-Pringle, C, Bradford, L, Reed, M, Baines, S, Fresque-Baxter, J, Jardine, T, et al. 2021, ‘Evaluating transdisciplinary research practices: Insights from social network analysis’, Sustainability Science, vol. 16, pp. 631–45. https://doi.org/10.1007/s11625-020-00901-y Stem, C, Margoluis, R, Salafsky, N & Brown, M 2005, ‘Monitoring and evaluation in conservation: A review of trends and approaches’, Conservation Biology, vol. 19, no. 2, pp. 295–309. https://doi.org/10.1111/j.1523-1739.2005.00594.x Tremblay, C 2017, Impact assessment: Community-Engaged Research (CER) at the University of Victoria 2009–2015, viewed 22 November 2021, https://dspace.library.uvic.ca/handle/1828/8166 Trow, M 1996, ‘Trust, markets and accountability in higher education: A comparative perspective’, Higher Education Policy, vol. 9, no. 4, pp. 309–324. https://doi.org/10.1016/S0952-8733(96)00029-3 Tyndall, D, Kosko, D, Forbis, K & Sullivan, W 2020, ‘Mutual benefits of a service-learning community-academic partnership, Journal of Nursing Education, vol. 59, no. 2, pp. 93–96. https://doi.org/10.3928/01484834-20200122-07 Van Tulder, R, Seitanidi, M, Crane, A & Brammer, S 2016, ‘Enhancing the impact of cross-sector partnerships: Four impact loops for channeling partnership studies’, Journal of Business Ethics, vol. 135, no. 1, pp. 1–17. https://doi. org/10.1007/s10551-015-2756-4 Wiek, A, Farioli, F, Fukushi, K & Yarime, M 2012, ‘Sustainability science: Bridging the gap between science and society’, Sustainability Science, vol. 7, pp. 1–4. https://doi.org/10.1007/s11625-011-0154-0 Williamson, H, Young, B, Murray, N, Burton, D, Levin, B, Massey, O & Baldwin, J 2016, ‘Community-University Partnership for Research and Practice: Application of an Interactive and Contextual Model of Collaboration’, Journal of Higher Education Outreach and Engagement, vol. 20, no. 2, pp. 55–84. Worton, S, Loomis, C, Pancer, S & Nelson, G 2017, ‘Evidence to impact: A community knowledge mobilisation evaluation framework’, Gateways: International Journal of Community Research and Engagement, vol. 10, pp. 121–42. https://doi.org/10.5130/ijcre.v10i1.5202 Plummer, et al. Gateways: International Journal of Community Research and Engagement, Vol. 15, No. 1 June 202217 https://doi.org/10.1016/j.jvb.2002.11.001 https://doi.org/10.1016/j.jvb.2002.11.001 https://doi.org/10.1016/S0149-7189(03)00029-6 https://doi.org/10.1016/S0149-7189(03)00029-6 https://doi.org/10.1016/j.techfore.2016.12.013 https://doi.org/10.1016/j.techfore.2016.12.013 https://doi.org/10.1080/13583883.2017.1395904 https://doi.org/10.1007/978-981-15-1940-6 https://doi.org/10.1007/s11625-020-00901-y https://doi.org/10.1111/j.1523-1739.2005.00594.x https://dspace.library.uvic.ca/handle/1828/8166 https://doi.org/10.1016/S0952-8733(96)00029-3 https://doi.org/10.3928/01484834-20200122-07 https://doi.org/10.1007/s10551-015-2756-4 https://doi.org/10.1007/s10551-015-2756-4 https://doi.org/10.1007/s11625-011-0154-0 https://doi.org/10.5130/ijcre.v10i1.5202