Microsoft Word - PPLT Vol. 5 Schechtel et al. (2022) (73269) FINAL.docx *Corresponding author – shauna.schechtel@ucalgary.ca Schechtel, S., Carpenter, Y., & Mozol, V. (2022). Shifting values and voices: An exploration in holistic mentorship evaluation. Papers on Postsecondary Learning and Teaching, 5, 108-115. SHIFTING VALUES AND VOICES: AN EXPLORATION IN HOLISTIC MENTORSHIP EVALUATION Shauna Schechtel*, Yuen-ying Carpenter, and Vivian Mozol University of Calgary The roles in traditional mentoring dyads are well known across both academic and professional contexts (Dawson, 2014). Despite the universality of these relationships, the way mentorship is evaluated in these relationships is fractured. Evaluation is limited to singular voices, singular points in time and simplified metrics to capture the journey and the unique experience of mentorship. These gaps push mentorship evaluation to try to encapsulate mentorship as a generalizable experience to satisfy metrics rather than acknowledging the dynamic complexity of these relationships. An exploration of current mentorship evaluation within the literature will highlight current limitations. These limitations allowed the authors to propose a new Co-Analysis model for evaluation that centers on shifting mentorship towards the values of partnership, flexibility, and holistic assessment. The model not only provides a universal pathway to improve any individual mentoring relationship, but also the opportunity for new voices to shape our understanding in future literature. Many forms of mentorship exist (Barrette-Ng et al., 2019), but none is more ubiquitous than the dyad mentor-mentee relationship. In this traditional relationship, a dyad is formed wherein the mentor is considered to have more experience and knowledge compared to their mentee (Barrette-Ng et al., 2019). The explicit valuation of the mentor’s relevant experience immediately sets up a power dynamic between the mentee and mentor based on the difference in status (Mullen & Klimaitis, 2021), as is the case between a student and their faculty mentor, or between junior and senior faculty members. Here, we propose a framework of priorities and values co-created by mentoring pairs in genuine partnership with each other. We define partnership in a mentoring context to stretch beyond simply collaborating to complete a task. Within the scope of this paper, partnership implies that each member of a mentorship pair is an equally valuable contributor, despite their differing entry points into the relationship – a stark contrast to the traditional dyad dynamics. In viewing mentorship as a partnership there is an expected shared responsibility between the mentor and mentee. Shared responsibility in a mentorship relationship means each member of the mentoring dynamic has the responsibility to work towards the desired outcome, to grow and shape how the mentorship relationship is evaluated. Each partner’s voice matters at all stages of the relationship, as both the mentee and mentor’s experiences have value and both parties are expected to grow as a result. We distinguish this form of partnership from ‘peer mentorship’ (a distinct form of mentorship not addressed here), as we are specifically interested in dyads where there is a significant difference in the status or experience each participating partner brings into the relationship (Barrette-Ng et al., 2019, Nowell et al., 2017). Schechtel et al. (2022) 109 In evaluating the success of such a partnership, we further wish to focus on formative and ongoing evaluation processes and more flexible metrics of success. Evaluation of mentorship should recognize the uniqueness of the individuals and relationship that is developed over the course of time, rather than being limited by strict role-based definitions (Dawson, 2014) and structures (Kram, 1983; Mullen & Klimaoitis, 2021). With these key ideas in mind, we begin by identifying some challenges in evaluating the complex nature of any mentoring relationship. Indeed, evaluation of the success of mentor-mentee dyads has been frequently studied, but the existing frameworks for evaluating mentorship face significant challenges. The utility of existing evaluations is limited by the way voices are constrained within strict boundary roles, by the types of evaluation metrics used, and by the timing of single-point evaluation. Challenge 1: Whose Voice Matters in Mentorship Many evaluations of mentoring success limit their focus to only one voice within the mentor-mentee dyad, including studies of both structured mentoring programs and individual relationships between graduate students and their faculty advisors (Eby et al., 2006; Tenenbaum et al., 2001). Mentees’ voices dominate the existing literature, wherein successful mentorship is solely measured by the growth of the mentee during their experience. Where prioritizing mentee perspectives in post-relationship evaluation aims to address the potential power imbalance in the relationship, mentee-only evaluations may, in fact, serve to reinforce the power dynamics of the expert/non-expert relationship. A narrow focus on the mentee problematically assumes a largely unidirectional transfer of advice, such that only the mentee is expected to learn from their relationship (Lee et al., 2015). Evaluating only what the mentee has gained from their mentor reinforces strictly defined roles within the relationship and consequently devalues the contributions of the mentee toward their mentor’s growth. If the mentee’s contributions to the relationship are not inherently valuable, then the decision-making power rests largely with the mentor. While this power differential may already exist outside of the partnership, mirroring these dynamics within the relationship itself can harm mentees who already struggle with a large gap in status compared to their mentor (Beyene et al., 2002; Goerisch, 2019). Some authors (Lee et al., 2015) suggest that mentees can advocate for themselves to have their voices heard during the relationship. However, such a task is inherently in competition with rigidly defined mentor-mentee roles, where the mentor’s experience provides them with the power to refuse or invalidate mentee needs (Goerisch et al., 2019, Tenebaum et al., 2001). It is, therefore, insufficient to put the mentee’s voice in focus only in summative evaluations and allow the mentor’s voice to dominate the events during relationship. While some studies bring the mentor’s voice into focus, it comes at the expense of losing the mentee’s (Eby et al., 2006). Evaluations that reflect only one voice in the relationship are inherently incomplete because we only get one side of a complex dynamic. No existing work on mentorship evaluation accounts for both voices in the relationship. Literature on cooperative learning which exists outside of the scope of the focus of the paper, may provide valuable insight to shape the exploration of voice in mentorship (Bruffee, 1995; Romer & Whipple, 1991). Without both voices, we are ignoring at least one key player in the process; we cannot make judgement calls related to the success, strengths, weaknesses, and health of any mentoring relationship without this information. Schechtel et al. (2022) 110 Challenge 2: Metrics Used in Mentorship Evaluation The largest variance in mentorship literature is in the evaluation metrics studied. Kram (1983) initially proposed a division of mentorship metrics into two broad categories: career growth and psychosocial growth (Mullen & Klimaitis, 2021). Career growth metrics measure how the relationship has helped the mentee reach goals such as networking and promotion or advancement (Kram, 1983; Mullen & Klimaitis, 2021). Psychosocial growth metrics track emotionally driven outcomes such as the mentee’s identity development or sense of belonging within their discipline or professional community (Palmer et al., 2015; Pfund et al., 2016). Researchers have continued to expand evaluation metrics into more specific categories to better reflect our understanding of mentorship, including research development, interpersonal development, and cultural diversity (Cramer & Prentice-Dunn, 2007; Pfund et al., 2016). While the categories of the metrics have been redefined, the types of evaluation metrics have not changed significantly and are ill-aligned to measure success of a given partnership. First, most metrics have been developed for formal programs of mentorship, limiting their utility to specific types of mentorship relationship such as peer mentoring (Mullen & Klimaitis, 2021) or graduate student-faculty (Tenenbaum et al., 2001). Moreover, program-level metrics inherently value institutional or program goals over the individuals’ experiences within the relationship (Goerisch, 2019; Palmer et al., 2015). For example, a mentorship program might be regarded as successful based on the retention of students within a discipline or degree (Chang et al., 2014). Such population-level statistics do not capture or describe the richness or challenges for each student’s relationship with their mentor (or mentee). While institutional priorities are crucial to continued funding and support for formal programs, institutional metrics should never control or evaluate the inherently human relationship of a given mentor-mentee pair. Second, most evaluations rely on readily quantifiable metrics collected through surveys or questionnaires (Ng et al., 2020, Tenenbaum et al., 2001) and disseminated through easily summarized statistics. Yet, many readily quantified metrics are ill-suited to describing the dynamics of a mentor-mentee relationship. For example, in seeking to evaluate communication in a partnership, a survey might ask the number of times a mentor-mentee pair met over the course of a year (Ramanan et al., 2005). While readily counted, this metric fails to evaluate the quality of the communication in those meetings – e.g. does the mentee feel heard in these meetings? For a more complete picture, one might ask the mentee, “How are your ideas being valued in our meetings?” or ask the mentor, “How comfortable do you feel sharing difficult experiences?” In examining the proposed alternatives, these deeper qualitative questions go beyond answering yes or no and asks the participants to reflect on their contributions and boundaries in the relationship. Challenge 3: Timing of Mentorship Evaluation Current studies evaluating mentoring limit their scope to either the beginning or the end of the relationship. Studies that focus on the start of the relationship emphasize the importance of alignment between the mentor and mentee’s beliefs. For example, formal mentorship programs often look for strong alignment through mentee pre-surveys (Rose, 2003) to ensure that mentor and mentee will work well together (Dawson, 2014). Metrics and questions often focus on clearly defining the goals and roles, as exemplified by asking a mentee to rate the importance of a statement such as, “My ideal mentor advocates for me in my research and career” (Rose, Schechtel et al. (2022) 111 2003). In contrast, evaluation that focus on post-relationship outcomes tend to ask for summative feedback on the “success” of the relationship (Tenenbuam et al., 2001). In many cases, authors used their work to create an itemized checklist of the behaviours of great mentors or mentees (Cramer & Prentice-Dunn, 2007; Dawson, 2014). In both cases, however, these evaluations are limited, as they attempt to describe a dynamic relationship over time by solely assessing a single moment. Pre-assessments argue that we can predict success as long the mentor-mentee pair meet certain checklist criteria for compatibility (Schrubbe, 2004). Yet, such a prediction does not account for experiences of the individuals in the relationship (Goerisch, 2019; Ng et al., 2020) nor how those experiences might trigger growth or change. Post-assessments offer feedback to both partners which they can take forward into new mentoring relationships, but do not allow the mentor or mentee to adapt responsively within their own partnership. Much like the single-point pre-assessment, the post- relationship evaluation is a snapshot which does not capture how the mentorship relationship changed over its duration. Post-evaluations may also suffer from bias as they ask individuals to remember nuance and detail, often forgotten over time in favour of highlighting the most memorable successes (Ng et al., 2020). These “snapshot” evaluations offer an incomplete understanding of a changing relationship (Ng et al., 2020). While there is a desire in the literature to complete a study that examines multiple points in a mentorship relationship, such work has yet to be completed (Ng et al., 2020). While evaluation over time increases the complexity of such a study, this complexity is at the center of the mentoring dynamic and therefore deserves exploration. Shifting Values in Mentorship Evaluation: A Co-Analysis Framework Relationship initiation frameworks (Barrette-Ng et al., 2019) and feedback in formal programs (Parise & Forret, 2008) have been developed to improve modern mentorship. Yet, neither of these tools offers a sufficient shift to address the gaps in voice, timing and alignment outlined above. We propose that addressing these gaps requires the use of a Co-Analysis framework for mentoring evaluations centered on three core values: partnership, flexibility, and holistic assessment. Defining mentorship as a partnership changes whose voice matters and how responsibility is accounted for in mentorship. While mentoring partners may not have equal experience, they can be equal partners in shaping and defining the mentorship relationship. Partnership as a core value does not just simply shift how power is distributed in the mentorship relationship, but rather removes the idea of power altogether in favoring a unified voice. Beyond merely labelling the relationship as a partnership, embodying this shift requires partners commit to shared responsibility and shared ownership. Valuing flexibility in mentoring evaluations acknowledges that each mentoring relationship is unique and that each relationship will evolve with time. As such, generalizable metrics designed for broad usage in pre- or post-assessment should not carry as much weight as the questions or metrics that are co-created during the specific mentoring partnership. Allowing partners to check-in with each other and re-assess their needs over time allows for the mentorship to develop and change responsively. We use the term holistic assessment here to mean the valuation of the bigger picture mentoring journey, or assessment over time. With this definition, a holistic framework can be understood as one that focuses on formative feedback that contributes to an iterative and Schechtel et al. (2022) 112 evolving partnership. In this feedback, we move away from asking for a time-independent checklist of what made the mentoring relationship successful and begin to ask how the partners can actively contribute to relationship success. Taken together, shifting toward these three core values address the gaps in mentorship evaluation literature, not by redefining our understanding of such relationships, but rather, by shifting what our relationships prioritize. Values Transformed into Action – Implementing the Co-analysis Framework To enact these values in practice, we emphasize that the Co-Analysis framework as a lens for the evaluation of any mentoring relationship. Critically, these values are not a checklist and should look different within the context of each individual mentoring partnership. Below (Figure 1) is one practical timeline for how the proposed values could be implemented in a mentoring relationship. Figure 1 Co-Analysis model steps that contain direct link to the new values for mentorship. The value can be indicated by the line type and colour at each step While each of the values is represented in the proposed timeline, the value that carries the most weight is partnership. Without this foundational value, we believe the mentoring relationship will suffer from unequal power and poor accountability. Both partners must be equally committed to re-evaluating and re-shaping their mentorship relationship. The partners ensure accountability to this value by ensuring both partners’ voices are heard, both are held accountable for their actions, and both demonstrate their willing to make concrete changes. Flexibility is demonstrated by the lack of a regimented schedule for evaluation and the openness to new reflection questions based on evolving evaluation needs. The key takeaway is that the evaluation metrics should be matched to the needs of the partners’ and their relationship at that. The cyclic nature of the proposed evaluation timeline reminds us of the holistic nature of our assessment. Unlike a linear model, where evaluation is clearly defined at a specific point in the mentorship, this cycle can be implemented and revised at any moment in the relationship. Schechtel et al. (2022) 113 To support practical implementation of these values, we also offer some guidance on the construction of reflection metrics. Depending on the needs of the partnership at a given time, we suggest discussion questions oriented to the following themes: • Goal questions focus on the direction or outcomes of the relationship journey; • Approach questions focus on the mentoring environment, including setting boundaries on your time, needs, and level of vulnerability; • Reflection questions center on the self, specifically one’s role in the relationship; and, • Feedback questions aim to constructively assess one’s mentoring partner. Again, these themes serve only as a guide, rather than a one-size-fits-all checklist. Not all partnerships are best served by all themes at all points in their unique and developing mentoring relationship. The authors acknowledge that this open-ended framework offers new priorities and is yet to be studied formally. Further, we recognize that this model introduces increasing complexity with a multi-point and ongoing evaluation framework. However, we contend that these values offer a broader, more human, more dynamic lens with which to account for the voices and relationships that are silent or limited in the current literature on evaluation. Conclusion The values of partnership, flexibility, and holistic assessment are the proposed path to a better mentorship that reflects the complexity of a real relationship rather than forcing it to fit a checklist of conditions. Placing the values of our model in the experience of the relationship we propose a Co-Analysis model to capture all voices at all stages of the mentoring relationship. The cyclic iterative nature pushes past the need to measure success at a start or end point because the importance is placed on the growth and journey each person takes. Every mentoring relationship matters and is essential to have metrics that capture a complete and authentic story rather than creating an incomplete picture to satisfy metrics, timing, and voices that are already well established in the literature. References Barrette-Ng, N., Nowell, L., Anderson, S.J., Arcellana-Panlilio, M., Brown, B., Chalhoub, S., Clancy, T.L., Desjardine, P., Dorland, A.M., Dyjur, P., Mueller, K., Reid, L., Squance, R., Towers, J., & Wilcox, G. (2019). The mentorship guide for teaching and learning. Taylor Institute for Teaching and Learning Guide Series Beyene, T., Anglin, M., Sanchez, W., & Ballou, M. (2002). Mentoring and relational mutuality: Proteges perspective. Journal of Humanistic Counselling, Education and Development, 41(1), 87-102. https://doi.org/10.1002/j.2164-490X.2002.tb00132.x Bruffee, K.A. (1995). Sharing our toys: Cooperative learning versus collaborative learning. The Magazine of Higher Learning, 27(1), 12-18. https://doi.org/10.1080/00091383. 1995.9937722 Chang, M.J. Sharkness, J., Hurtado, S., & Newman, C.B. (2014). What matters in college for retaining aspiring scientists and engineers from underrepresented racial groups. Journal of Research in Science Teaching, 51(5), 555-580. https://doi.org/10.1002/tea.21146 Cramer, R.J., & Prentice-Dunn, S. (2007). Caring for the whole person: Guideline for advancing undergraduate mentorship. College Student Journal, 41(4), 771-778. https://eric.ed.gov/?id=EJ816801 Schechtel et al. (2022) 114 Dawson, P. (2014). Beyond a definition: Towards a framework for designing and specifying mentoring models. Educational Researcher, 43(3), 137-145. https://doi.org/10.3102/ 0013189X14528751 Eby, L.T., Durley, J.R., Evans, S.C., & Ragins, B.R. (2006). The relationship between short-term mentoring benefits and long term mentor outcomes. Journal of Vocational Behavior, 69(3), 424-444. https://doi.org/10.1016/j.jvb.2006.05.003 Goerisch, D., Basiliere, J., Rosener, A., McKee, K., Hunt, J., & Parker, T.M. (2019). Mentoring with: Reimagining mentoring across the university. Gender, Place & Culture, 26(12), 1740-1758. https://doi.org/10.1080/0966369X.2019.1668752 Kram, K.E. (1983). Phases of the mentor relationship. Academy of Management Journal, 26(4), 608-625. https://doi.org/10.2307/255910 Lee, S.P., McGee, R., Pfund, C., & Branchaw, J. (2015). The mentoring continuum: From graduate school through tenure. In R. Wright, G (Eds.), “Mentoring Up”: Learning to manage your mentorship relationships (pp. 133-153). Graduate School Press of Syracuse University. Mullen, C.A., & Klimaitis, C.C. (2021). Defining mentoring: A literature review of issues, types, and applications. Annals of the New York Academy of Sciences, 1483(1), 19-35. doi:10.1111/nyas.14176 Nowell, L., Norris, J.M., Mrklas, K., & White, D.E. (2017). A literature review of mentorship programs in academic nursing. Journal of Professional Nursing, 33(5), 334-344. https://doi.org/10.1016/j.profnurs.2017.02.007 Ng, YX., Koh, Z.Y.K., Yap, H.W., Tay, K.T., Tan, X.H., Ong, Y.T., … Krishna, L. (2020). Assessing mentoring: A scope review of mentoring assessment tools in internal medicine between 1990-2019. PLOS ONE, 15(5), 1-18. https://doi.org/10.1371/journal.pone.0232 511 Palmer, R.J., Hunt, N.A., Neal, M., & Wuetherick. (2015). Mentoring, undergraduate research, and identity development: A conceptual review and research agenda. Mentoring & Tutoring: Partnership in Learning, 23(5), 411-426. https://doi.org/10.1080/ 13611267.2015.1126165 Parise, M.R., & Forret, M.L. (2008). Formal mentoring programs: The relationship of program design and support to mentors’ perceptions benefits and costs. Journal of Vocational Behaviour, 72(2), 225-240. https://doi.org/10.1016/j.jvb.2007.10.011 Pfund, C., Bayars-Winston, A., Branchaw, J., & Eagan, K. (2016). Defining attributes and metrics of effective research mentoring relationships. AIDS and Behavior, 20, 238-248. https://doi.org/10.1007/s10461-016-1384-z Ramanan, R.A., Taylor, W.C., Davis, R.B., & Phillips, R.S. (2006). Mentoring matters: Mentoring and career preparation in internal medicine residency training. Journal of General Internal Medicine, 21(4), 340-345. https://doi.org/10.1111/j.1525- 1497.2006.00346.x Romer, K.T., & Whipple, W.R. (1991). Collaboration across the powerlines. College Teaching, 39(2), 66-70. https://doi.org/10.1080/87567555.1991.9925490 Rose, G.L. (2003). Enhancement of mentor selection using the ideal mentor scale. Research in Higher Education, 44(4), 473-494. https://doi.org/10.1023/A:1024289000849 Schrubbe, K.F. (2004). Mentorship: A critical component for professional growth and academic success. Journal of Dental Education, 68(3), 324-328. https://doi.org/10.1002/j.0022- 0337.2004.68.3.tb03748.x Schechtel et al. (2022) 115 Tenenbaum, H.R., Crosby, F.J., & Gliner, M.D. (2001). Mentoring relationships in graduate school. Journal of Vocational Behaviour, 59 (3), 326-341. https://doi.org/10.1006/jvbe.2001.1804