Microsoft Word - Larson et al. 2022 Publish.docx Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 91 Selecting and Implementing a Telementoring Program: Case Studies of Project ECHO Rebecca Sam Larson and Caryn Medved 1 Diffusion Associates, East Lansing, MI, 2 Department of Communication Studies, Baruch College, City University of New York Cite as: Larson, R.S. & Medved, C. (2022). Selecting and implementing a telementoring program: Case studies of project ECHO. Metropolitan Universities, 33(3), 91-109. DOI: 10.18060/25689 This is an open access article distributed under the terms of the Creative Commons Attribution License. Guest Editors: Joseph A. Allen, Ph.D., Sara Hart, Ph.D., and Melissa Yack, Ph.D. Editor: Valerie L. Holton, Ph.D. Abstract Extension for Community Healthcare Outcomes (Project ECHO) is a telementoring program for health professionals that uses adult learning techniques and interactive video technology to connect distal community providers with specialist and multidisciplinary teams in real-time collaborative sessions. We examine the adoption, implementation, and sustainability of ECHO programs at four academic medical centers through case studies based on structured interviews. The study and its analysis are informed by the Diffusion of Innovation theory and the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. We found that early adopters became aware of ECHO by chance and were persuaded through observations to adopt ECHO. Finding a home for ECHO was an important initial adoption decision. Five context factors influence the implementation of ECHO: Funding, networks, staffing processes, leadership, and individual characteristics of staff. Sustainability requires ongoing funding, which itself may rely on evidence of outcomes. Findings from this study can inform the implementation of Project ECHO at other academic medical centers and extend to decisions to adopt, implement, and sustain similar telementoring programs designed to close the research- practice care gap between communities and academic medical centers. Keywords: telementoring, project echo, implementation science, diffusion of innovations Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 92 Introduction Dr. Sanjeev Arora, a Hepatitis C specialist at the University of New Mexico Health Sciences Center, walked into his clinic to find a 43-year-old woman with Hepatitis C seeking treatment for the first time after her initial diagnosis eight years earlier. When asked why she delayed treatment, she said she could not afford to take time off of work to make the five-hour trip to Albuquerque. She finally sought help when her abdominal pain began interfering with her ability to work. Now it was too late. The untreated Hepatitis C had caused advanced liver cancer that was not suited for surgery or liver transplantation. Guidelines and medicine to treat this patient’s illness were available, but the doctor in her community did not have the expertise required to treat her disease. She died six months later (AJMCtv, 2018). The knowledge-practice gap in medicine is often described as a twofold challenge: Clinicians are required to learn new knowledge and evidence-based practices and learn how to use those practices in their day-to-day work (Price, 2005). But the challenge is three-fold in academic medical centers seeking to fill this gap. Educational interventions are typically the means to address the knowledge-practice gap. Selecting and implementing an educational intervention is an additional challenge. How do staff at academic medical centers find educational interventions designed to close the research-practice gap in distal communities of care? What factors influence the implementation of these interventions? How are such interventions sustained? These are the research questions we explore in this paper by looking at four case studies of one telementoring intervention – Project ECHO. This paper first describes Project ECHO, then discusses the conceptual framework that guided our work, and details how we conducted this study. Next, we present findings and conclude with a discussion of how these findings transfer to other academic medical centers seeking to implement Project ECHO and similar telementoring programs designed to close the research- practice care gap. Project ECHO Dr. Arora recognized that rural health care providers could help their patients with complex conditions if they had additional support from specialists such as himself. As a result, he created the telementoring program Extension for Community Healthcare Outcomes (Project ECHO). Project ECHO uses adult learning techniques and video technology (e.g., Zoom) to connect distal community health providers with medical specialists and multidisciplinary teams in real-time collaborative sessions. The ECHO idea is straightforward. The knowledge traditionally held by Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 93 specialty care providers moves out to the community to uptrain generalist providers so they can maximize treatment before a patient needs to be referred to a specialist, if at all. ECHO allows patients to get the care they need from their local provider, who they regularly seek care from and who knows them, and the local context. As a result, care is more timely and more personal. It also frees up specialists’ time to focus on more complex patients. Each ECHO site has an operational hub from which virtual telementoring programs are created, advertised, managed, and evaluated. ECHO hubs can be situated in an academic medical center, like those we profile in this paper. Hubs also reside in health care systems, national associations, state agencies, and nonprofit organizations. A hub can be a free-standing unit that focuses only on ECHO programs. Hubs can also be embedded within a unit, such as a continuing education office, where ECHO is one of several “tools in the toolbox.” Hubs can also comprise a loose collection of individuals who coordinate their resources to provide ECHO programs. A hub, then, consists of a team of people who offer ECHO programs to participants. The hub team may include one or several specialists or experts who are physicians, medical researchers, advanced care nurses, clinical pharmacists, or other persons with specialized knowledge. It also includes administrative staff who provide operational support for the program. Those attending ECHO sessions are typically individual providers—family physicians, nurses, nurse practitioners, physician assistants, social workers, and others. Participants are seeking to learn more about a condition, treatment, process, or policy to better meet the needs of their patients in local communities. Participants are recruited through emails, hub websites, listservs, flyers, and professional networks. Participants may be grouped as cohorts and participate in the same sessions. Alternatively, participation may be open and fluid, with participants coming and going at will. Participants can often accrue free continuing education credits through participation. ECHO programs focus on health conditions (e.g., chronic pain, Hepatitis C, HIV/AIDS) and healthcare (e.g., quality improvement, nursing homes, community health workers). Topics are selected to reflect the interests of the participants as identified through focus groups, literature, surveys, and personal knowledge. ECHO programs consist of telementoring sessions facilitated by hub experts or specialists. Sessions are designed to encourage “all teach, all learn” (Arora et al., 2017), where learning is the process of constructing new knowledge on the foundation of existing knowledge (Mukhalalati & Taylor, 2019). This is achieved through brief expert-led didactics and a case presented by a participant or an expert. Patients are commonly the subject of cases, but patients are not participants. Policies and procedures may also be subjects of cases. Cases may be submitted in advance or constructed through discussions. After a case is presented, the session facilitator first invites participants to comment or ask clarifying questions and then Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 94 asks for recommendations and impressions. At the close of the discussion, the facilitator and other experts or specialists offer their impressions. The facilitator summarizes the discussion. ECHO refers to this learning design as “telementoring.” The goal of Project ECHO is a “democratization of knowledge” so that patients anywhere can receive the best care from the doctor or health care practitioner they can easily access. Project ECHO moves knowledge, not people (Arora et al., 2017). By creating environments where “all teach, all learn,” participants learn about new evidence and recommended practices from specialists and peers who may have similar challenges. Specialists at academic medical centers learn about patients they may never see and local conditions that shape how, or if, care is delivered. Project ECHO began in 2003 with a Hepatitis-C program at the University of New Mexico. Currently, there are more than 370 ECHO hubs and more than 2,600 programs offered in the United States and Canada. Worldwide, ECHO has spread to more than 650 hubs in 58 countries (Project ECHO, 2022). Several studies find that the ECHO approach enhances access to medical treatments by helping primary care providers in underserved areas (Dearing et al., 2019; Tran et al., 2021). Much of ECHO’s growth in the United States is due to its adoption and implementation in academic medical centers. Project ECHO is well suited to academic medical centers. Education is a primary mission of academic medical centers as they educate the next generation of physicians and provide continuing education for current providers. Physicians in these centers have expertise based on research and caring for complex patients. Academic medical centers are also deeply embedded in their communities and may serve as a safety-net provider. As can happen with a fast-spreading innovation, the state of everyday practice may go undescribed. That is, what we confidently know about how the innovation is being implemented and sustained is delayed as science needs to catch up. Funded by the Robert Wood Johnson Foundation, we conducted interviews in late 2020 to study Project ECHO's adoption, implementation, and sustainment across multiple sites intending to fill this knowledge gap. Specifically, we explored factors that influence decision-making about ECHO. Findings from this study can inform the adoption of Project ECHO at other academic medical centers and extend to similar telementoring programs for community-based health care providers. Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 95 Conceptual Foundation We frame the decision to adopt, implement, and sustain Project ECHO using Diffusion of Innovation theory and Implementation Science literature. Diffusion is how an innovation, such as ECHO, is communicated through certain channels over time among the members of a social system (Rogers, 2003). For an innovation perceived to be important, an individual’s decision to adopt an innovation is not an instantaneous act; it is a process that occurs over time (Rogers, 2003). Typically, the adoption decision process begins when potential adopters become aware of the innovation; they gain knowledge through exposure to descriptive information about what problem the innovation addresses and how it functions. Knowledge may come from passive or accidental exposure or through an intentional effort to seek out a solution to a known problem. Potential adopters may seek information on using the innovation or the principles that underlie it. If the knowledge is perceived as relevant and adequate, adopters reach a persuasion stage where they seek evaluative information to reduce uncertainty about the consequences of the innovation. They want to understand better the advantages and disadvantages of the innovation and, often, who else has already adopted it. The next stage is the decision to adopt or reject the innovation. Potential adopters may try out the innovation during this phase or observe trial adoption by a peer (Rogers, 2003). If a decision is made to adopt, implementation begins as the innovation is used. Failed implementation efforts are often the underlying reason that best or promising practices are ineffective in health and social care systems and organizations (Moullin et al., 2019) and educational settings (Nordstrum et al., 2017). Implementation Science is the study of methods to promote an understanding of and find solutions to the cause of variation in program outcomes to improve the quality and effectiveness of interventions (Eccles & Mittman, 2006). The goal of implementation science is not to establish the impact of an innovation but to identify the factors that affect its uptake into routine use (Bauer & Kirchner, 2020). Most research about the organizational implementation of this type has occurred in healthcare settings such as clinics and hospitals. Many implementation science frameworks have been proposed and tested, such as the Consolidated Framework for Implementation Research (CFIR) to identify and categorize independent variables that affect the implementation of evidence-based practices (Damschroder et al., 2009) and the Promoting Action on Research Implementation in Health Services (PARIHS) framework to predict why implementation will or will not be effective (Harvey & Kitson, 2016). We use the Exploration, Preparation, Implementation, Sustainment (EPIS) framework (Aarons et al., 2011) to guide our study. The EPIS framework was developed based on implementing innovations in public sector social and allied health services and has been applied in educational settings (Moullin et al., 2019; Movsisyan et al., 2019). The EPIS Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 96 framework organizes factors that influence implementation into four constructs: Outer and inner context factors, bridging factors, and innovation factors. Outer context factors describe the environment external to the organization. In contrast, inner context factors refer to characteristics within an organization. Bridging factors cross between outer and inner contexts, and innovation factors focus on the characteristics of the innovation (Aarons et al., 2011). In this study, we were particularly interested in outer and inner context factors that influence implementation. We did not look for all EPIS inner and outer factors as some did not fit with this study (e.g., patient characteristics). We chose not to focus on innovation factors because the ECHO model is not prescriptive but based on principles and adaptation is anticipated as contexts are different. The factors we looked for in this study are defined in Table 1. Table 1. Study definitions of epis outer and inner context factors Context EPIS Factor Study Definition Outer context Funding Fiscal support provided by the system in which ECHO occurs Inter-organizational environment and networks Relationships through which knowledge of ECHO is shared and/or implementation goals are established Service environment State and federal sociopolitical and economic contexts that influence the implementation of ECHO Inner context Organizational staffing processes The processes or procedures related to the hiring, training, and retention of staff involved in ECHO implementation Leadership Characteristics and behaviors of individuals involved in oversight and/or decision-making related to ECHO implementation within an organization Individual characteristics of staff Characteristics of individuals that influence the process of ECHO implementation Quality and fidelity monitoring/support Processes or procedures to ensure adherence to active delivery of the ECHO Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 97 Organizational characteristics Structures or processes in an organization that may influence the process of ECHO implementation Note: This table is drawn from factors and definitions in Aarons et al., 2011 and Moullin et al., 2019. Study Design and Analysis Procedures We conducted case studies that describe the adoption, implementation, and sustainability of ECHO programs at four academic medical centers. Using a qualitative case study design (Stake, 1995), at each site, we interviewed 2-3 people associated with the ECHO hub and 2-4 people associated with two separate ECHO programs. Programs at each hub were selected based on how they were different. A most-different approach to selection maximizes structural differences and attends to similarities in observation despite the inherent differences across cases (Przeworski & Teune, 1970). As is typical of case study research, we primarily collected information about each case through an interview protocol (Brewer & Hunter, 1989). Interviews followed a structured protocol, and all respondents were asked the same open-ended questions designed to learn how the hub came to adopt Project ECHO and the contextual factors that shape program implementation and sustainability. Respondents included medical and administrative leaders who are part of the ECHO hub at each site. We also interviewed medical experts and administrative coordinators for each ECHO program. Interviews were conducted via Zoom, recorded, and transcribed. Case studies were constructed through an iterative process of comparing and contrasting responses from each site based on guiding questions (Miles et al., 2014). Each case was reviewed by respondents who validated the content. Next, the authors conducted a cross-case comparison. According to Miles et al. (2014), one of the key reasons to conduct cross-case analysis is to “deepen understanding and explanation” (p. 101). Using Roger’s (2003) innovation decision- making model and EPIS factors (Aarons et al., 2011) as sensitizing concepts (Charmaz, 2003), authors individually compared and contrasted data across these four cases and then came together to discuss insights that emerged from the cross-case analysis. The sites we include in this study are as follows: ECHO Colorado, University of Colorado Anschutz Medical Campus; ECHO Utah, University of Utah Health; ECHO Nevada, University of Nevada, Reno School of Medicine; ECHO Chicago, University of Chicago Medicine. Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 98 Findings ECHO Model Adoption-Decision Process In this section, we discuss how adopters of the ECHO model gained knowledge of ECHO, what influenced their decision to adopt it, and where they implemented it in the organization. In each of the four academic medical centers we studied, awareness of Project ECHO came from serendipitous exposure. Dr. Box at the University of Utah Health first learned of Project ECHO in 2010 from a colleague who had been informed about the development of ECHO by a pharmaceutical representative. Dr. Johnson, who was instrumental in bringing ECHO to the University of Chicago, learned about ECHO in 2009 from a colleague who had learned about ECHO through a friend who worked at the University of New Mexico. At the annual American Academy of Medical Colleges meeting in 2014, the President of the University of New Mexico asked the Chancellor of the University of Colorado, “Why don’t you have an ECHO?” In 2012, two physicians at the University of Nevada, Reno School of Medicine, read about ECHO in an article in the New England Journal of Medicine written by Dr. Arora. In each of these cases, the adopters were not specifically searching for a telementoring intervention to address a practice- care gap. This is not an uncommon occurrence. Needs may be developed after one becomes aware of an innovation (Rogers, 2003). Upon learning about ECHO, leaders from each academic medical center independently visited the ECHO Institute at the University of New Mexico. There was limited training and support infrastructure during these early years of the Project ECHO movement, so much was learned directly from Dr. Arora and his team, including observing an ECHO session led by Dr. Arora. These site visits were highly impactful. One physician stated, “I saw what was going on, including the tele-clinic. I got the introduction to the philosophy and the platform and the full potential impact. I returned determined to recreate it.” Leaders returned from visits to the ECHO Institute ready to implement Project ECHO. One of their first implementation decisions was where to locate ECHO within their respective academic medical center. Project ECHO presents a bit of a dilemma in terms of organizational fit. It is an educational intervention. It is a form of community engagement. It involves technology. It may involve a single specialty or engage a multidisciplinary team across departments or even colleges. It is not surprising then to find that ECHO programs did not share a common location with an academic medical center – or that these homes may change over time. Below we describe where ECHO fits within each case study site. Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 99 ECHO Nevada is housed within the Office of Statewide Initiatives at the University of Nevada, Reno School of Medicine. The Office of Statewide Initiatives is charged with improving access to quality health care for rural Nevada by providing collaborative leadership and resources to health care and community organizations. The fit, then, is with the mission of an office within the School of Medicine. ECHO Chicago is situated within the University of Chicago’s Biological Sciences Division in the Department of Pediatrics. Within this Department, the hub is part of the Academic Pediatrics Section, one of two sections led by Dr. Johnson – who brought ECHO to the University of Chicago and continues to provide leadership. ECHO Chicago also aligns with Dr. Johnson’s work with an Urban Health Initiative focused on improving the delivery of care in urban, underserved communities. Fit coalesces around Dr. Johnson’s expertise and interests. In Colorado and Utah, the location of ECHO work has changed over time. At both sites, ECHO work initially aligned with the interests of the adopter. In Colorado ECHO, the initial physician leader had an appointment in the School of Public Health, and ECHO was initially located there. This leader retired, and the ECHO programs shifted to be more clinically focused, which enabled ECHO Colorado to diversify its funding. Eventually, the ECHO work merged with the campus eConsult program. This led to ECHO Colorado transitioning from the School of Public Health to the School of Medicine. This shift in location is pragmatic and, at least in part, based on funding opportunities. At the University of Utah Health, ECHO Utah was initially supported by the Senior Vice President for Health Science and the Departments of Medicine and Surgery chairs. The initial framing of ECHO Utah was as an educational intervention and as a business growth opportunity. Dr. Box, who brought ECHO to Utah, defined ECHO as an educational program, and ECHO was aligned with his work in the transplant service line. Several years later, the ECHO portfolio of programs was moved to the Office of Network Development and Telehealth and placed within the Education Team. The shift in Utah is similar to Colorado in that it reflects a pragmatic need to find alignment not just with the mission or an individual’s interests but also with funding. Returning to the innovation-decision process model (Rogers, 2003), what we learn across these four cases is that initial awareness of ECHO was serendipitous and generated a perceived need to learn more about it. A critical incident at the stage of persuasion for these early-adopting sites was meeting with Dr. Arora and his staff at the ECHO Institute. Once the decision to adopt ECHO was made, leaders had to choose where to initially place an ECHO hub within the academic medical center. Across these four case studies, decisions about placement were shaped Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 100 by three factors: the adopter’s expertise, the alignment of ECHO with the mission of a particular unit, and/or funding opportunities. Project ECHO Implementation and Sustainability Factors The EPIS framework identifies outer (system) factors and inner (organizational) factors that have been found in previous studies to impact the implementation and sustainability of innovations (Aarons et al., 2011). No two academic medical centers describe the same set of factors as influencing the implementation of ECHO. Still, they do coalesce around five factors: Funding, Inter-organizational Environment and Networks, Organizational Staffing Processes, Leadership, and Individual Characteristics of Staff. Funding is the single factor that consistently surfaced related to the sustainability of Project ECHO. Below we illustrate how each of these factors influences the implementation and sustainability of ECHO at each study site. Implementation Factors Funding A key characteristic of the ECHO model is that participants do not pay to attend ECHO programs. Offering the program at no cost to participants facilitates the democratization of knowledge as participation is not contingent on an ability to pay. ECHO hubs and programs must secure financial resources to support their work, and this is an ongoing challenge that influences how or if an ECHO program can be offered. At ECHO Colorado, the funding model was described as “braided… a very diversified funding model.” This comment well characterizes the funding at all four ECHO sites. Funding for ECHO typically came from five sources. Pass-through funds were issued by a federal agency to a state agency or institution who then awarded these funds to an academic medical center for ECHO work. Pass-through funds came as grants with a programmatic focus, such as opioid use disorder or COVID, and were made on an annual or bi-annual basis, though they may last for several years. The second source of funds was from or tied to Medicaid. Like pass-through funds, Medicaid-related funds came tied to a specific focus or condition. A third type of funds came from research grants where ECHO was typically not the primary focus but part of a bundle of interventions. Again, these funds were associated with a specific topic or workstream and were for a limited period. These three sources of external funds were a critical part of the budget but seldom supported all of the ongoing operations of an ECHO hub. Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 101 Institutional funds from the academic medical center or the university can bring a much-needed cushion to programs. These internal funds may function as recurring funds, even if they are not, and provide support for program development, marketing, evaluation, and research. Institutional support was common, but it was seldom in abundance. For example, ECHO Utah received institutional funds, but these funds were “just enough” to keep current programs running. A final source of funds was foundations grants or gifts. Foundation funds were fewer in number, but they had significant impact. For example, the Colorado Health Foundation invested more than $3 million to jump-start ECHO Colorado. This investment allowed for a thoughtful approach to the role of ECHO Colorado and hiring an experienced director, coordinators, and learning specialists. Looking for funding to support programmatic and operational needs was “exhausting.” One respondent summed up the experience described at each study site: “A critical part of the job is looking for money. We all know that, eventually, this kind of money dries up because you’re no longer as innovative as you once were, and so people don’t always want to keep funding you.” Funding was critical to implement ECHO. Academic medical centers had unique funding “braids” and sought to diversify their funding sources. Inter-Organizational Environment and Networks Academic medical centers operated ECHO in a network of professional relationships, and these relationships influenced the implementation of ECHO. Physician leaders reached out to safety net organizations, primary care groups, and state hospital associations, among other medical groups, to identify community needs, share information about ECHO, and recruit participants. Dr. Johnson at ECHO Chicago explained that he went to a care collaborative to “identify the safety net organizations that would be interested in hearing about ECHO, and then leveraged my relationships with FQHCs [Federally Qualified Health Centers] to launch ECHO.” ECHO hubs at these four academic medical centers worked with state health and related services departments. State officials played multiple roles – funder, expert, connector, certifier, and thought partner. Respondents at three academic medical centers mentioned having advisory boards that include members from external organizational partners. Community-based organizations were also frequent partners. These organizations – some of which provided health and some of which offered safety nets for underserved populations – helped to identify community needs, connected the ECHO programs to participants, and sometimes provided experts to deliver the didactic portion of an ECHO session. Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 102 ECHO hubs were also reaching across their respective academic medical centers to identify faculty who could serve as facilitators, team members, or presenters. ECHO hub directors partnered with campus marketing, communication, and evaluation offices. Several respondents reported that ECHO program collaborations led to new partnerships and multidisciplinary research proposals. In a couple of situations, the ECHO hubs partnered with outreach and external affairs offices and were profiled as key outreach programs for the academic medical center. Organizational Staffing Processes Professional training in implementing the ECHO model was an essential factor identified at each of the four case sites. Nearly all of the ECHO hubs and programs staff attended a three-day “immersion” training provided by the ECHO Institute at the University of New Mexico. The immersion training provided an opportunity to see an ECHO program in action, making the process more tangible. Dr. Thomas at ECHO Colorado attended immersion training multiple times. Each time he traveled to New Mexico, he looked at ECHO from a unique perspective; “I went to look at it [ECHO] from an entrepreneurial perspective, I went from a technology perspective, I went from a healthcare delivery perspective. I tried to listen to it differently and interact with different people at different times.” Staff were encouraged to attend in teams, which many did. At ECHO Nevada, sending teams improved knowledge of how to implement ECHO and also created buy-in to the purpose of ECHO. The impact of the training extended beyond learning “how” to do ECHO. Respondents referenced “drinking the Kool-Aid” and finding camaraderie and shared purpose with attendees from their own and other institutions. At ECHO Utah, one respondent described immersion in a way similar to what we heard at other sites: “When you go to immersion and get fully immersed in everything ECHO, it gives you a different drive and a different understanding of what’s going to work and what’s not going to work.” Respondents, and especially physicians, shared that working in clinics and their professional training influenced how they approached ECHO implementation. Experience as health practitioners in low-resource settings, community settings, and telehealth guided many physicians in their approach to ECHO programs. Physicians also relied on their substantive knowledge and expertise in the topic of the ECHO program. Some staff, though not all staff, drew on experiences in quality improvement training and their education in public health. Training, especially immersion training at the ECHO Institute, was important to adhere to the model's fidelity. But the impact of immersion training went beyond “how to do” ECHO, it inculcated a shared sense of “why to do” ECHO. Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 103 Leadership Institutional leadership strongly influenced the decision to adopt Project ECHO and influenced where ECHO was located, at least initially, in each academic medical center. Leadership continued to be important during the implementation phase though leaders changed and how they influenced implementation varied. At ECHO Nevada, the hub’s medical director provided leadership of day-to-day operations and quality improvement. At the other three academic medical centers, medical directors continued to champion the work but much, if not all, of the leadership of operations, fell to administrative leaders. In ECHO Colorado and ECHO Chicago, the administrative leaders had a broad scope of responsibilities with upwards of ten direct reports. Leadership was also evident at the program level, where medical experts facilitated sessions that encouraged a learning environment where “all teach, all learn.” Program leaders were described as “modeling” openness and vulnerability. At ECHO Nevada, facilitators sometimes presented their patient cases and asked participants for review and consultation. This method showed that even subject matter experts did not have all the answers and can use input from others. Respondents at ECHO Colorado talked about the importance of the ECHO medical leader in mentoring facilitators, experts, and staff in developing a culture where everyone feels safe to share their challenges. At ECHO Chicago, one program facilitator talked about his role in setting “the tone” of the ECHO so that it was “about respect and seeing the people on the other end as equals. We’re all doing this together.” ECHO implementation required leaders of hubs to champion the work to other parts of the academic medical center and the university. However, they did not need to provide day-to-day leadership of operations. At the program level, leadership was expressed as an ability and willingness to model behaviors that build a safe space where participants were comfortable sharing their cases, asking questions of their peers, and providing advice to their peers and the subject matter experts. Individual Characteristics of Staff Successful implementation of ECHO programs relied on the commitment and knowledge of program coordinators. ECHO coordinators often supported four or more ECHO programs and worked closely with program facilitators. At ECHO Chicago, one program facilitator stated, “The most important person for all of this has been our project coordinator. That is somebody that can do the type of outreach that she’s done, run the sessions, and make sure that everything Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 104 happens. It just doesn’t work otherwise.” Facilitators relied on coordinators as mentors who gave a sense of “here’s the to-dos and not to-dos. Here’s how you talk about it. Here’s what’s helpful. Here’s how to brand it. Those basics really helped us get off to a confident, good start.” Coordinators also often linked the program team with the larger ECHO community or other stakeholder groups. They shared organizational knowledge about ECHO, its practices, values, and ethos. Across these four sites, facilitators and coordinators described close working relationships and respect for each other. Coordinators did more than administrative work; they were mentors to the facilitators, guiding them on how to work most effectively with participants and prepare for sessions. Funding as a Sustainability Factor The ECHO hubs and programs we studied anticipated growing their ECHO work over the next five years. Growth was often focused on increasing the number of participants attending programs and expanding into new geographic areas, delving into new topics (including moving outside of traditional medical areas), or increasing the number of cases from participants. Growth is not possible, nor is maintaining the status quo, if programs cannot be sustained over time. Funding was the one factor that emerged across these four sites as essential to maintain current and future ECHO work. Each site was searching for ways to expand its financial base and gain greater financial stability. Three sites specifically mentioned working with the state to secure dedicated funds for ECHO. All sites were writing proposals for funds, often in response to a request from the state for pass- through funds, and to local and national foundations. Sites were looking for funds that extended beyond specific topics so that they were free to evolve their ECHO programming. They were looking for more “protected time” for academic center medical staff to work on ECHO to offset costs and increase dedication. Financial stability could lead to longer-term commitments for administrative staff and investments in strategic planning, evaluation, and research. Sustainable funding was dependent, to some extent, on showing the value of ECHO to the university, the state, and others. Dr. Thomas at ECHO Colorado said, “We have to publish more than feel-good related articles, that we’ve done something with ECHO that is meaningful to our different partners.” A program leader at ECHO Colorado said, “We’re getting real results. People are doing things they weren’t doing before that align with best practices. If someone could study the program and show the evidence, then I could expand it.” ECHO Utah shared a similar sentiment, saying that they need to deliver a message to the state about “why we exist, why we matter, and how we’re making an impact for citizens across the state and region.” Some sites Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 105 were encouraging academic faculty to include ECHO in research grant proposals which could increase the likelihood that ECHO outcomes are being measured. Each ECHO hub collected routine process and outcome data as required by the ECHO Institute or continuing medical education credits. These data, however, fell short of showing impacts. Being part of an academic medical center did not guarantee access to researchers or evaluators. Finding staff who could conduct evaluations or analyze existing data was a need expressed by three ECHO hubs. A strategy to obtain longer-term funding is to invest in research that provides evidence that ECHO work is reducing health care disparities and or reducing health care costs. Conclusions and Recommendations ECHO hubs were well established in these four academic medical centers. Awareness of ECHO came from serendipitous encounters. Observations and conversations led to adoption. Finding a home for ECHO was an important initial implementation decision. Implementation of ECHO programs was shaped by funding, networks, training, leadership, and individual characteristics of staff. Sustainability is a consistent concern and focus. Attendees do not pay to participate in ECHO programs; thus, funding must come from elsewhere. ECHO hub leaders seeking funds from their respective states, foundations, and within their institution would be well served by having evidence of the impact of their work. What can we learn from these four ECHO hubs that can inform the adoption and implementation of ECHO at other academic medical centers? For one, those curious should attend immersion training to see firsthand how ECHO programs and sessions are organized and delivered. These immersive trainings are very impactful and could lead to adoption or non-adoption decisions. Second, there is no one ideal place where ECHO should be housed within an academic medical center, which is not surprising as each center has a unique organizing structure. Fit is important with a faculty-physician-researcher’s work or with the mission of a larger division or office, and fit may change over time. Third, ECHO programs are a form of community engagement as they cultivate partnerships with and respond to the needs of the larger community as well as within the organization. Fourth, the staff in the ECHO hubs are as critical as the medical experts. They are the glue that keeps programs moving forward. Beyond their administrative and coordination roles, they also train specialists in an “all teach, all learn” model of learning. Lastly, and most importantly, funding for these programs has to be addressed from the beginning. ECHO programs are not revenue-generating and require external and internal funding sources to implement and sustain them. Some programs have been discontinued at these successful ECHO Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 106 hubs due to a lack of funding or shifts in funding priorities. Providing “protected time” for academic center medical staff to work on ECHO is one way to offset costs. Much of what we are learning from the implementation of ECHO hubs at these four academic medical centers can apply to the adoption, implementation, and sustainability of other telementoring programs. One key translational finding is the role of serendipity. What if these physicians had not had the one-off conversation or happened across a journal article? We can be more intentional in finding effective innovative solutions to enduring problems. For example, we can provide medical experts and staff with time and support to attend conferences or events, especially those outside of their disciplinary expertise. Academic medical centers can also designate offices or individuals with systematic scanning of literature and conferences proceedings to identify promising interventions. Another translational finding is that the factors identified in this study that influence implementation and sustainability are not novel or unique to ECHO – we find these factors influencing program implementation in many health systems (Aarons et al., 2011). If an academic medical center – or other organization – adopts a similar educational program, it should be attending to these factors from the start. For example, if a program is not revenue- generating or doesn’t generate sufficient revenue to sustain the staff and other programmatic costs, finding stable funding should be a primary focus. As another example, when teams are going to implement a new program, having the opportunity to train together can foster fidelity to the model. Related, training should focus on “know why” and not only “know how.” In addition, embedding programs in professional and community networks can strengthen the ties between medical experts and those in practice who are serving diverse populations. Networks also help with recruiting participants and guest speakers. Another point to consider is that before COVID-19, the use of video technology such as Zoom was novel or at least unique in medical education. That is no longer the case. Project ECHO has kept from being just another webinar because of its continued focus on “all teach, all learn.” Facilitating technology-based learning requires a new set of skills for many physicians and staff in academic medical centers. In addition to medical specialty knowledge, skills are needed in adult learning, facilitation, case-based learning, and peer-based collaboration. Finally, while Project ECHO draws on partnerships and has generated new collaborations, the reach across the university to colleges in social science, business, and education, among others, and lifelong learning units is weak and should be strengthened. The social determinants of health that create the conditions that foment health disparities need a wide-angle lens to solve them. Inviting other disciplines to the work of Project ECHO could strengthen its impact. Too, lifelong Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 107 learning efforts across campus could benefit from more knowledge about Project ECHO’s highly elastic model for sharing best practices and mastering complex knowledge. We believe that the ECHO model could be widely used across disciplines. Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 108 References Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health, (38), 4-23. https://doi.org/10.1007/s10488-010-0327-7 AJMCtv. (2018, Nov 2). Dr Sanjeev Arora explains the origins of Project ECHO and improving access to care [Video]. YouTube. Retrieved from https://www.youtube.com/watch?v=nw8Sx3aMLx8 Arora, S., Kalishman, S. G., Thornton, K. A., Komaromy, M. S., Katzman, J. G., Struminger, B. B., Rayburn, W. F., & Bradford, A. M. (2017). Project ECHO: A telementoring network model for continuing professional development. The Journal of Continuing Education in the Health Professions, 37(4), 239–244. https://doi.org/10.1097/CEH.0000000000000172 Bauer, M. S. & Kirchner, J. (2020). Implementation science: What is it and why should I care? Psychiatry Research, (283), Article 112376. https://doi.org/10.1016/j.psychres.2019.04.025 Brewer, J. & Hunter, A. (1989). Multimethod research: A synthesis of styles. Thousand Oaks, CA: Sage. Charmaz, K. (2003). Grounded theory: Objectivist and constructivist methods. In Denzin, N. K. & Lincoln, Y. S. (Eds.), Strategies for qualitative inquiry (2nd ed., pp. 249–291). Thousand Oaks, CA: Sage Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, (4), Article 50. https://doi.org/10.1186/1748-5908-4-50 Dearing, J. W., Cruz, S., Kee, K., Larson, R. S., & Rahm, A. K. (2019). Project ECHO: Review and research agenda. Final report to the Robert Wood Johnson Foundation. East Lansing MI: Diffusion Associates. Retrieved from http://www.diffusionassociates.com/wp- content/uploads/2019/09/echo.pdf Eccles, P. E. & Mittman, B. S. (2006). Welcome to Implementation Science. Implementation Science, (1), Article 1. https://doi.org/10.1186/1748-5908-1-1 Harvey, G. & Kitson, A. (2016). PARIHS revisited: From heuristic to integrated framework for the successful implementation of knowledge into practice. Implementation Science, (11), Article 33. https://doi.org/10.1186/s13012-016-0398-2 Original Research © The Author 2022. Published by the Coalition of Urban and Metropolitan Universities. www.cumuonline.org Metropolitan Universities | DOI 10.18060/25689 | June 11, 2022 109 Miles, M. B., Huberman, A. M, & Saldaña, J. (2014). Qualitative data analysis: A methods sourcebook. Thousand Oaks, CA: Sage. Moullin, J. C., Dickson, K. S., Stadnick, N. A., Rabin, B., & Aarons, G. (2019). Systematic review of the exploration, preparation, implementation, sustainment (EPIS) framework. Implementation Science, (14), Article 1. https://doi.org/10.1186/s13012-018- 0842-6 Movsisyan, A., Arnold, L., Evans, R., Hallingberg, B., Moore, G., O’Cathain, A., Pfadenhauer, L.M., Segrott, J., & Rehfuess, E. (2019). Adapting evidence-informed complex population health interventions for new contexts: A systematic review of guidance. Implementation Science, (14), Article 105. https://doi.org/10.1186/s13012-019- 0956-5 Mukhalalati, B. A. & Taylor, A. (2019). Adult learning theories in context: A quick guide for healthcare professional educators. Journal of Medical Education and Curricular Development, (6), 1-10. https://doi.org/10.1177/2382120519840332 Nordstrum L. E., LeMahieu, P. G., & Berrena, E. (2017). Understanding and finding solutions to variation in program implementation. Quality Assurance in Education, 25(1), 58-73. https://doi.org/10.1108/QAE-12-2016-0080 Price, D. (2005) Continuing medical education, quality improvement, and organizational change: Implications of recent theories for twenty-first-century CME. Medical Teacher, 27(3), 259-268. https://doi.org/10.1080/01421590500046270 Project ECHO. (2022, January 18). Data on our work. Retrieved February 1, 2022. Retrieved from https://hsc.unm.edu/echo/partner-portal/data-marketplace/ Przeworski, A. & Teune, H. (1970). The logic of comparative social inquiry. New York: John Wiley & Sons. Rogers, E. M. (2003). Diffusion of innovations. New York: Free Press. Stake, R. E. (1995). The art of case study research. Thousand Oaks, CA: Sage. Tran, L., Feldman, R., Riley, T. & Jung, J. (2021). Association of the Extension for Community Healthcare Outcomes Project with use of direct-acting antiviral treatment among US adults with Hepatitis C. JAMA Open, 4(7): e2115523. https://doi:10.1001/jamanetworkopen.2021.15523