This work is licensed under a Creative Commons Attribution 4.0 International License. Increasing Social Work’s Stake in Interprofessional Practice: Effectively Preparing MSW Students for Integrated Care Settings Amy Benton Alejandro Allen Lindsey Anderson Erica Nason Abstract: Integrated behavioral healthcare is becoming increasingly important and central to social work practice as the healthcare field shifts from a solely disease model of health to a more holistic approach. Researchers have also noted the need for studies demonstrating effective models for incorporating integrated healthcare training into schools of social work. Studies that demonstrate the effects of integrated training have the potential to provide legitimacy to these training programs, an important step in furthering their adoption within schools of social work in the United States and abroad. The training program described here was designed to increase the number of MSW social workers who provide trauma-informed, culturally relevant, evidence-based behavioral health prevention and intervention practices at integrated healthcare settings. This study used a pre- and post-survey design with non-participating MSW students as a control group. Findings indicate significant increases in knowledge, skills, and attitudes for training participants. A recommendation for further research is to explore to what extent graduates transfer their enhanced knowledge to the workplace. Keywords: Integrated healthcare, social work education, BHWET programs, stipend- supported programs, quantitative method Integrated behavioral healthcare is becoming increasingly important and central to social work practice as the nature of the healthcare field shifts from a solely disease model of health to a more holistic approach (Fraher et al., 2018; Zerden et al., 2018). This increasingly holistic approach calls for integrated teams of professionals from different specializations to come together to improve outcomes for individuals and communities. As a result, social work education needs to enhance and support the training of MSW students to work effectively in interprofessional healthcare settings. This study provides positive results of one such enhanced MSW training project. Literature Review Integrated Healthcare Practice Examples of professions included on interprofessional teams include medicine, nursing, occupational therapists, speech therapists, psychologists, social workers, among others. However, barriers have been identified for incorporating social work into interprofessional practice, including the need to fight for legitimacy in integrated healthcare settings, misunderstandings of what social work has to offer, social workers’ _________ Amy Benton, PhD, Associate Professor, Alejandro Allen, MA, MSW student, Lindsey Anderson, LCSW, Lecturer, and Erica Nason, PhD, Assistant Professor, School of Social Work, Texas State University, San Marcos, TX. Copyright © 2022 Authors, Vol. 22 No. 3 (Fall 2022), 976-992, DOI: 10.18060/25624 about:blank Benton et al./INCREASING SW STAKE IN IPE 977 wide breadth of practice and skills, and difficulties associated with working in environments that still heavily rely on the medical model (Ashcroft et al., 2018; Fraher et al., 2018; Tucker & Webber, 2020; Zerden et al., 2020a). Social workers’ training and socialization to their professional field imbue them with strengths that can be added to interprofessional teams to improve outcomes for clients. In particular, social work’s focus on the social determinants of health and their importance for achieving equitable outcomes across populations, the ecological perspective, and systems theory have resulted in an understanding that the healthcare field “requires social work’s expertise to promote and renew health and wellness in the lives of individuals, families, groups, and communities as well as to impact policy development” (Rubin et al., 2020, p. i). Social workers’ understandings of the social determinants of health is particularly valuable for interprofessional teams, as integrated practice is being recognized as a way of effectively addressing healthcare disparities and achieving more positive outcomes for people of color (Davis et al., 2015; Stanhope et al., 2015). Other areas of social work training that are highly relevant in interprofessional settings include group facilitation, culturally competency, family work and engagement, and ethics (Jones & Phillips, 2016). Additionally, social workers are trained in other hard and soft skills that can be impactful for interprofessional teams and fill important gaps. For example, Craig et al. (2020) found that social workers add value to integrated healthcare teams through their active communication skills, ability to educate others, and their familiarity with risk management strategies. Zerden and colleagues (2020a) found that MSW students who were placed in interprofessional settings for their field placements recognized a lack of communication, conflict management, and decision-making skills in the interprofessional teams they worked on. These three skills are important skills that social workers receive training on throughout their formal education experiences and beyond, meaning that social workers are efficiently equipped to improve the functioning of the interprofessional healthcare teams on which they work. Integrated Care Training A large number of social workers already work in integrated healthcare settings (Padykula et al., 2020). However, while integrated care education models have been well- developed in fields such as nursing, social work integrated care education models are comparatively younger and less developed (Zerden et al., 2018). Increasing social work’s stake in interprofessional practice requires improved education and training for social work students before going into the field post-graduation (Zerden et al., 2018). Where integrated healthcare education is being incorporated into social work curricula, evaluation results indicate that these efforts show demonstrable increases in feelings of preparedness to work in integrated healthcare settings (Acquavita et al., 2020; Mattison et al., 2017; Zerden et al., 2020b). Social workers require training in many areas to be competent interprofessional practitioners, such as team-based care, cultural competency, inter-team communication, trauma-informed care, and evidence-based practices for physical health assessments ADVANCES IN SOCIAL WORK, Fall 2022, 22(3) 978 (Horevitz & Manoleas, 2013; Mattison et al., 2017; Padykula et al., 2020). In one study, social workers working in integrated care settings in Canada identified specialized education and training to be vital for improving the social worker’s integration into integrated healthcare teams (Ashcroft et al., 2018). However, healthcare professionals like social workers are not frequently provided opportunities for formal interprofessional education in their university programs (Zerden et al., 2020a). Past research demonstrates that social workers frequently lack formal training in integrated healthcare competencies skills and service delivery models (Padykula et al., 2020). Instead, many social workers report learning the skills that they use on the job without formalized training programs (Horevitz & Manoleas, 2013). In one study, only 10% of participants reported learning the interprofessional skills they use from their MSW program (Horevitz & Manoleas, 2013). This finding is significant, as it points to a continuing need for trained social workers who are arriving in professional practice already possessing skills and knowledge they need to practice effectively. Formal, specialized training during their education would not only prepare practitioners to be more effective when arriving to new positions but would also ensure that they are all consistently trained in the same integrated healthcare competencies. Improving social work education models, and in particular field education, related to integrated care is essential for improving social workers’ ability to work, and competitiveness, in integrated healthcare settings (Fraher et al., 2018; Horevitz & Manoleas, 2013; Padykula et al., 2020; Zerden et al., 2017). Field education provides students opportunities to use the skills they have learned in real-world situations and provides crucial training that can only be achieved through hands-on experience with real people and populations. Additionally, it provides students opportunities to experience working on interprofessional teams and helps them understand the skills necessary to work on such teams. However, training social workers in integrated care is limited by the number of field placements in these settings and the availability of field instructors who can supervise students in these settings. As such, efforts need to be made to identify more integrated healthcare field placements for students to practice in (Mattison et al., 2017; Padykula et al., 2020; Putney et al., 2017; Zerden et al., 2020b). Still, increased field placements in integrated healthcare settings may not be sufficient for increased efficacy after graduation. Researchers have also noted the need for studies demonstrating effective models for incorporating integrated healthcare training into schools of social work (Mattison et al., 2017). Studies that tangibly show the effects of integrated training have the potential to provide legitimacy to these training programs, an important step in furthering their adoption within schools of social work in the United States and abroad. Stipend Supported Training Programs Stipend supported training programs are effectively being used in social work education to increase students’ knowledge and skills and better prepare them for practice in their respective subfields. The stipend-supported programs present in the literature reflect varied approaches to providing education beyond the MSW curriculum, including workshops, seminars, specialized and relevant coursework, and specialized field Benton et al./INCREASING SW STAKE IN IPE 979 placements (Acquavita et al., 2020; Foster et al., 2013; Putney et al., 2017; Sampson, 2017; Zerden et al., 2017). The majority of the literature on social work stipend-training programs focuses on programs related to child welfare work, such as the Title IV-E and National Child Welfare Workforce Institute (NCWWI) training programs. Both programs have consistently demonstrated increases in knowledge, increased feelings of preparedness for working in child welfare, and higher levels of retention in child welfare work following completion of the program (Benton & Iglesias, 2018; Benton et al., 2020; Gansle & Ellett, 2002; Leake et al., 2015; O’Donnell & Kirkner, 2009). However, studies examining these dimensions are limited, as they have also generally suffered from a lack of a control group to understand whether the results seen can be attributed to the stipend-supported programs. An exception to this is a study by Bagdasaryan (2012), who found that Title IV-E MSW students scored significantly higher (p=.01) on knowledge tests than MSW students who did not participate in the Title IV-E program. Behavioral healthcare stipend-supported training programs are beginning to gain traction in social work education. For example, the Global Leaders of Behavioral Health Education (GLOBE) training project provides participants with monthly stipends, specialized training, and field placements in behavioral health with the end goal of preparing graduates for work in behavioral healthcare settings (Sampson, 2017). Another program, the California Social Work Education Center’s Mental Health Program (MHP), provided behavioral health training to students across 17 social work programs with a focus on increasing retention in the behavioral health field. In a post-graduation study, Foster et al. (2013) found that 93.7% of those who participated in the MHP program remained employed in the behavioral health field. The authors credit curriculum development, financial incentives, and program evaluation as key factors in the success of the program (Foster et al., 2013). Other behavioral healthcare stipend-supported training programs have specifically focused on training students for practice in integrated settings, largely due to the Health Resources Services Administration (HRSA) recognizing the need and providing substantial funding. The Behavioral Health Workforce Initiative (BHWI) provided stipends to MSW students in exchange for completing clinical electives and interprofessional seminars (Putney et al., 2017). Evaluation of the program showed increased knowledge about, and confidence in, working in integrated healthcare settings across two cohorts of students (Putney et al., 2017). However, this study did not use a pre- and posttest design, instead using a retrospective posttest design, which researchers note may have resulted in inflated assessment of levels of change (Putney et al., 2017). The University of North Carolina’s PrimeCare (UNC-PrimeCare) program trained social workers to practice in integrated behavioral healthcare settings by focusing on interprofessional education and field experience. Zerden and colleagues (2017) published findings evaluating the program, though the findings primarily focused on the process of developing new field placements in integrated care settings and the extent to which students were able to experience interprofessional collaboration at their field site. Smith-Osborne and Daniel (2017) demonstrated that undergraduate and graduate participants in a stipend- supported integrated behavioral healthcare training program scored highly on tests demonstrating knowledge of integrated behavioral healthcare content, increased their sense ADVANCES IN SOCIAL WORK, Fall 2022, 22(3) 980 of themselves as interprofessional team members, and were successful in using integrated behavioral healthcare concepts learned in the classroom. In a cross-site study examining growth in interprofessional knowledge, skills, and values among participants in a stipend supported training program, Acquavita and colleagues (2020) found that MSW-level students showed an increase in their self- perceived ability to work with others, value in working with others, and comfort in working with others following participation in the program. This study’s findings are limited, as it did not include a control group to draw conclusions about the effectiveness of the program. Another study by Zerden and colleagues (2020a) used a quasi-experimental matched group design to compare MSW students who had field placements in integrated healthcare settings and participated in a training program to those who had field placements in integrated healthcare settings but did not participate in the program. Results of this study varied. There was no significant difference between the two groups on a scale measuring participants’ attitudes toward collaborative care in integrated care settings. Yet, results also indicated that students in the training program improved at a significantly higher rate on a scale assessing team collaboration skills (Zerden et al., 2020a). Further studies are needed to examine the effectiveness of programs which prepare social workers for integrated behavioral healthcare practice. More needs to be known regarding their attitudes, knowledge, and skills in working with diverse individuals, primarily in economically disadvantaged communities. Additionally, despite a growing knowledge base of published research on behavioral health and integrated behavioral healthcare training programs, these findings often suffer from the same methodological pitfalls as the studies examining child welfare training programs, as they do not include control groups to accurately assess whether the changes seen result from the training program itself. Research incorporating control groups are needed to provide key insights and help demonstrate the effectiveness of these programs in a more concrete manner. Project Description This project was funded through a Behavioral Workforce Education & Training (BHWET) grant from HRSA, and was housed in a School of Social Work at a large public university located in the southwest United States. It was designed to enhance students’ knowledge, skills, and professional development by providing a variety of learning opportunities in addition to their regular MSW education. The goal of the project was to increase the number of trained social workers who provide trauma-informed, culturally relevant, evidence-based behavioral health prevention and intervention practices at schools, hospitals/clinics, and homeless serving agencies. The project recruited, educated, and trained 110 graduate-level social work students over a four-year period. Participating students committed to working with vulnerable populations and in underserved areas in integrated behavioral health care and community settings. Students also participated in experiential learning modules, training sessions, and related behavioral health workshops prior to and during the final field semester. These Benton et al./INCREASING SW STAKE IN IPE 981 learning opportunities furthered their knowledge regarding effectively serving vulnerable populations and underserved areas. The modules were designed to build behavioral health related knowledge and skills when working in integrated behavioral health settings. Participants were organized into cohorts who collectively participated in the project’s enhanced learning opportunities in the semester prior to and during their final field semester of the graduate program. The intervention remained consistent across the project years. Specific hypotheses guiding evaluation of the project were as follows: H1. Project participants will reflect greater levels of knowledge, skills & abilities in project specific practice competencies (e.g., working in integrated care settings, trauma-informed care) than the control group. H2. Project participants will reflect greater levels of cultural competency than the control group. Method The researchers used a two-group pre-post design to evaluate the effectiveness of the project. Quantitative evaluation data were collected at the initiation and completion of the project for each cohort in the training condition, with concurrent collection of survey data from control group members. All study participants completed surveys assessing their knowledge, skills, attitudes, and practices related to key project competencies based on standardized/validated measures described below. The study was approved by the Texas State University’s IRB. Recruitment and Sample Approximately three months prior to the beginning of each project cycle, emails were sent to MSW students who were projected to be eligible for final field in two semesters’ time, advertising the opportunity to participate in the project. Interested students filled out applications which were then reviewed by a committee consisting of the project director, the project coordinator, and a volunteer member of the field faculty. Initial criteria included a complete application, previous experience, and clearly stated interest in integrated behavioral healthcare. To learn more about potential students and their interest in the project, follow-up interviews with selected applicants began in Spring 2019 and continued through December 2021. Interviews provided an opportunity to screen for students who sincerely wanted more learning opportunities. Selected students were notified of their acceptance into the project and invited to an orientation session. The total number of students in the training group was 110. Each member of the training group received the $10,000 stipend as stipulated by the funders, with no additional compensation for participating in the research component. The quantitative analysis examines data for the sample that filled out both pre and post surveys (n=89), providing a response rate of 81%. Students who did not apply or were not accepted into the project were invited to be in the control group. Control group participants received a $20 egift card ADVANCES IN SOCIAL WORK, Fall 2022, 22(3) 982 each for pre and post survey. A total of 74 students initiated involvement as a control group member. The total number of control participants who completed both pre and post surveys was 52 (70%). The majority of students in the training sample were white, female, with ages 23-32. Similarly, the control group was primarily female and 23-32 years old. The control differed from the training group in that participants were roughly equal in terms of reporting white or Hispanic for ethnicity. Over a third of the sample came from economically disadvantaged backgrounds or rural settings. Table 1 provides sample demographics. Table 1. Training and Control Group Demographics Demographic Group n (%) Training (n= 89) Control (n= 52) Gender Female 81(91%) 43 (83%) Male 5 (6%) 7 (13%) Nonbinary 2 (4%) Missing 3 (3%) Ethnicity Asian-American 1(1%) 2 (4%) African American 11 (12%) 8 (15%) Hispanic 22(25%) 21 (40%) White 45 (51%) 19 (37%) Multi-ethnic 7 (8%) 2 (4%) Missing 3 (3%) Bilingual Yes 33 (37%) 20 (39%) No 53 (60%) 31 (60%) Missing 3 (3%) 1 (2%) Age 23-25 34 (38%) 20 (39%) 26-32 32 (36%) 21 (40%) 33+ 20 (23%) 10 (19%) Missing 3 (3%) 1 (2%) Rural Yes 33 (37%) 15 (29%) No 53 (60%) 36 (70%) Missing 3 (3%) 1 (1%) Economically Disadvantaged Yes 34 (38%) 20 (39%) No 51 (57%) 31 (60%) Missing 4 (5%) 1(2%) Veteran Status Yes 4 (4.5%) 3 (6%) No 81 (91%) 49 (94%) Missing 4 (4.5%) Benton et al./INCREASING SW STAKE IN IPE 983 Measures Two self-report surveys were employed to evaluate the project’s effectiveness in: 1) enhancing participant knowledge and skill attainment and 2) increasing understanding of attitudes and behaviors related to cultural competence. Paper versions of the pre-surveys were administered by the project coordinator during orientation sessions. Post-surveys were administered by the project coordinator approximately six months later, on the last day of field seminar. Due to COVID-19, changes were implemented to reduce the amount of face-to-face contact. Starting in March, 2020 and through project end, pre- and post- surveys were sent electronically to participants via a secure link through the web-based survey tool, Qualtrics. Knowledge, Skills, and Abilities Assessment (KSA). The knowledge, skills, and abilities instrument measured practitioners’ understanding and proficiency related to a series of competencies relevant to behavioral health prevention and intervention practices. The survey used was adapted from the Knowledge, Skills, and Abilities Assessment for Youth Practitioners (National Collaborative on Workforce and Disability, 2004) in order to better reflect the project focus. For example, all questions regarding disability were removed as this was not a focus of the project. The other adaptation was including a competency on trauma-informed care, as this topic was a key component of the expanded education provided through the project. Thus, the instrument assessed seven competency areas including 1) knowledge of the field, 2) communication with clients, 3) assessment and individualized planning, 4) relationship to family and community, 5) supervision, 6) administrative skills, and 7) trauma-informed care. Items on the survey consisted of sample statements included with each competency to ensure shared definitions when assessing self in regard to each competency. For example, under the trauma-informed competency, one of the statements was “Ability to help diverse clients make meaning of their trauma history and current experiences from a trauma-informed perspective”. There were a total of 24 items. Participants rated each item within each competency area on a 4-point Likert scale in knowledge, training, and relevance. Knowledge ratings ranked participants’ level of skill and knowledge related to the competency (e.g., 1=I currently know little to nothing about this; 2=I have some knowledge or skill in this; 3=I know a good amount about this, but it would be useful to learn more; 4=I am very knowledgeable or skilled in this). Training ratings ranked participants’ level of training in the selected competency (e.g., 1=I have received little to no training in this competency to date; 2= I have received some training in this competency to date; 3=I have received a lot of training in this competency to date; 4=I have received a certificate or degree in this competency). Relevance ratings ranked participants’ belief in how relevant each competency is to job responsibilities (e.g., 1=I believe this competency has little to no relevance to my job responsibilities; 2=I believe this competency is somewhat relevant to my job responsibilities; 3=I believe this competency is very relevant to my job responsibilities; 4=I believe this competency is critical to my job responsibilities). Means and standard deviations for the KSA are presented in Table 2. ADVANCES IN SOCIAL WORK, Fall 2022, 22(3) 984 Table 2. Means and Standard Deviations for Knowledge, Skills, and Abilities Assessment for Practitioners Competencies for Two Groups at Pre- and Post-Assessments Competency Domain M* (SD) Pretest Posttest Training Group (n= 82) Control Group (n= 45) Training Group (n= 82) Control Group (n= 45) Knowledge of the Field Relevance 3.86 (0.32) 3.67 (0.55) 3.91 (0.21) 3.67 (0.49) Knowledge 2.81 (0.41) 2.92 (0.53) 3.50 (0.43) 3.13 (0.48) Training 2.50 (0.53) 2.59 (0.54) 3.21 (0.59) 2.88 (0.57) Communication With Clients Relevance 3.91 (0.19) 3.69 (0.55) 3.94 (0.19) 3.71 (0.43) Knowledge 2.87 (0.50) 3.11 (0.62) 3.62 (0.46) 3.38 (0.44) Training 2.62 (0.55) 2.67 (0.66) 3.31 (0.56) 2.88 (0.63) Assessment and Individualized Training Relevance 3.12 (0.48) 3.13 (0.56) 3.18 (0.43) 2.98 (0.65) Knowledge 2.16 (0.55) 2.33 (0.73) 2.81 (0.54) 2.63 (0.66) Training 2.01 (0.65) 2.02 (0.76) 2.63 (0.56) 2.31 (0.67) Relationship to Family and Community Relevance 3.72 (0.45) 3.70 (0.58) 3.82 (0.33) 3.46 (0.63) Knowledge 2.61 (0.53) 2.89 (0.83) 3.40 (0.60) 3.05 (0.62) Training 2.31 (0.62) 2.45 (0.78) 3.09 (0.60) 2.68 (0.78) Supervision Relevance 3.88 (0.31) 3.72 (0.57) 3.98 (0.13) 3.74 (0.45) Knowledge 2.98 (0.74) 3.06 (0.72) 3.76 (0.43) 3.37 (0.54) Training 2.59 (0.73) 2.74 (0.80) 3.38 (0.60) 2.88 (0.83) Administrative Skills Relevance 3.92 (0.23) 3.79 (0.40) 3.93 (0.21) 3.72 (0.41) Knowledge 3.11 (0.47) 3.24 (0.49) 3.66 (0.43) 3.47 (0.46) Training 2.76 (0.58) 2.84 (0.55) 3.30 (0.49) 3.06 (0.58) Trauma Informed Care Relevance 3.90 (0.23) 3.75 (0.56) 3.92 (0.22) 3.55 (0.66) Knowledge 2.37 (0.63) 2.43 (0.78) 3.41 (0.56) 2.76 (0.75) Training 2.08 (0.71) 2.06 (0.89) 3.16 (0.65) 2.53 (0.92) * Range=1-4 for all competencies and domains Cultural Competence (CC). Participants also completed the Promoting Cultural and Linguistic Competency: Self-Assessment Checklist for Personnel Providing Primary Health Care Services (Goode, 1999). The CC is a self-assessment that includes three subscales assessing practitioners’ office environment, communication style, and values and attitudes in relation to cultural and linguistic competence. An example question from the office environment subscale reads, “I ensure that printed information disseminated by my agency or program takes into account the average literacy levels of individuals and families receiving services.” From communication style, an example is, “I understand the implications of health literacy within the context of my roles and responsibilities.” An example from the values and attitudes subscale reads, “Even though my professional or moral viewpoints may differ, I accept individuals and families as the ultimate decision makers for services and supports impacting their lives.” In the CC measure, participants responded to each item using a 3 point Likert scale (3=Things I do frequently, or statement applies to me to a great degree, 2 =Things I do occasionally, or statement applies to me to a moderate degree, or 1=Things I do rarely or never, or statement applies to me to a minimal degree or not at all). Both the KSA and CC Benton et al./INCREASING SW STAKE IN IPE 985 have good psychometric properties. Average Cronbach’s alphas for subscales of the CC instrument were .80 for Physical environment (4 items), .92 for Communication styles (13 items) and .83 for Values and attitudes (24 items). For the KSA instrument, the Cronbach’s alphas were .94 for Relevance, .90 for Knowledge/proficiency, and .93 for Training (all had 24 items). Means and standard deviations for the CC are presented in Table 3. Table 3. Means and Standard Deviations for Cultural Competencies Scale for Two Groups M* (SD) Pretest Posttest Training Group (n= 87) Control Group (n= 48) Training Group (n= 87) Control Group (n= 48) Physical Environment 1.91 (0.69) 2.00 (0.64) 2.31 (0.57) 2.15 (0.63) Communication 2.40 (0.46) 2.47 (0.51) 2.81 (0.26) 2.52 (0.46) Values and Attitudes 2.62 (0.29) 2.48 (0.42) 2.85 (0.17) 2.68 (0.28) * Range=1-3 for all scales Analysis SPSS 26 (IBM, 2019) was used to analyze the quantitative data. Subscales were created by summing the items and taking their average. The KSA had 21 subscales and the CC had three subscales. Items on the CC scale were recoded so higher scores indicate that participants do the things related to cultural competence more frequently. Mean differences on post survey responses were assessed using ANCOVA. Time (pre- versus post-test) was a repeated-measures factor in this model, and group (training versus control) was a between-subjects factor. The time-by-group interaction effect is key to testing whether the treatment is more effective than the control. A power analysis was conducted using the G*Power software package (Faul et al., 2007). In order to achieve a power of .80 to detect a treatment effect of medium size (f2=.15), 56 total participants would be required. We ran 24 ANCOVAs for each individual scale of the KSA and CC. In each ANCOVA, the covariate was the pre-survey subscale, the dependent variable was the post- survey subscale, and the independent variable was the grouping variable (training or control). ADVANCES IN SOCIAL WORK, Fall 2022, 22(3) 986 Table 4. Analysis of Knowledge, Skills, and Abilities Assessment With Pretest as the Covariate Relevance Knowledge Treatment SS df F p SS df F p SS df F p Competence One: Knowledge of the Field Pretest 2.33 1 23.82 <.001** 2.96 1 16.30 <.001** 2.53 1 7.93 .006* Treatment 0.85 1 8.67 .004** 4.80 1 26.48 <.001** 3.05 1 9.57 .002* Error 12.11 124 22.50 124 40.17 126 Competency Two: Communication With Clients Pretest 0.74 1 10.07 .002** 2.23 1 11.96 .001** 5.97 1 20.87 <.001** Treatment 0.71 1 9.66 .002** 2.08 1 11.15 .001** 6.23 1 21.77 <.001** Error 9.51 129 23.72 127 36.35 127 Competency Three: Assessment and Individualized Planning Pretest 17.98 1 133.39 <.001** 15.98 1 73.42 <.001** 16.15 1 78.04 <.001** Treatment 1.51 1 11.65 .001** 2.10 1 9.63 .002* 3.77 1 18.24 <.001** Error 16.63 128 27.43 126 26.07 126 Competence Four: Relationship to Family and Community Pretest 1.03 1 5.48 .021* 5.05 1 15.89 <.001** 5.55 1 14.16 <.001** Treatment 3.84 1 20.46 <.001** 5.64 1 17.74 <.001** 6.87 1 17.54 <.001** Error 24.41 130 41.03 129 50.14 128 Competency Five: Supervision Pretest 1.91 1 33.62 <.001** 2.41 1 11.51 .001** 5.78 1 13.74 <.001** Treatment 0.66 1 11.68 .001** 4.30 1 20.50 <.001** 9.14 1 21.72 <.001** Error 7.33 129 26.82 128 54.27 129 Competency Six: Administrative Skills Pretest 0.18 1 1.96 .164 1.87 1 10.38 .002* 3.20 1 13.47 <.001** Treatment 1.14 1 12.65 .001** 1.35 1 7.47 .007* 2.09 1 8.79 .004* Error 11.67 129 23.28 129 30.15 127 Competence Seven: Trauma-Informed Care Pretest 0.43 1 2.46 .12 4.75 1 12.90 <.001** 13.51 1 30.93 <.001** Treatment 3.15 1 18.07 <.001** 13.47 1 36.57 <.001** 13.89 1 31.80 <.001** Error 22.69 130 0.37 129 56.68 130 Note: *p < .05, ** p <.001 Benton et al./INCREASING SW STAKE IN IPE 987 Results For the KSA, each of the seven competencies were rated three times in the following areas: relevance, knowledge, and training. The ratings on individual items ranged from 1 to 4 with higher scores meaning the participants thought the competency had relevance, their knowledge and skills level for the competency was high, and that they had a lot of training in the competency. Results of the ANCOVAs revealed that the relevance, knowledge, and training ratings were statistically significant for all 7 competencies. Effect sizes ranged from small to medium. In all cases, the training group means were higher than the control group means indicating that participants in the training group thought each competency was more relevant, that they were more knowledgeable in all competencies, and they had more training in all competencies than the control group. Therefore, the first hypothesis was supported. Table 4 provides the statistical output for each subscale of the KSA assessment. For the CC assessment, results of the ANCOVA analysis, two of the three subscales were statistically significant: Communication Styles and Values and Attitudes. For the two statistically significant subscales, the training group had higher mean scores than the control group, indicating that they increased their cultural and linguistic behaviors and attitudes as a result of the training they received (statistically significant at p <.001). There was not a significant difference between groups for the Physical Environment subscale. These outcomes only partially support the second hypothesis. Table 5 provides the results for each subscale of the CC assessment. Due to the demographic differences between the training and control groups with regard to ethnicity, ANCOVA was used to assess for any demographic differences associated with scores on the KSA and CC. On the KSA, there were no demographic differences associated with Competencies 1-4 and 6-7. However, scores for non-White students were higher than for White students on the Relevance and Knowledge subscales of Competency 5 (Supervision; p=.022 and p=.034, respectively). There were no significant findings related to demographics for Treatment on Competency 5. There were no differences based on ethnicity for any of the three CC subscales. Table 5. Analysis of Cultural Competence Scale With Pretest as the Covariate Scale SS df MS F p Physical Environment Pretest 4.65 1 4.65 14.59 <.001** Treatment 1.11 1 1.11 3.48 .065 Error 42.04 132 0.32 Communication Pretest 2.18 1 2.18 20.76 <.001** Treatment 2.97 1 2.97 28.32 <.001** Error 12.05 115 0.11 Values and Attitudes Pretest 0.25 1 0.25 6.01 0.16 Treatment 1.00 1 1.00 23.57 <.001** Error 4.62 109 0.42 Note: ** p <.001 ADVANCES IN SOCIAL WORK, Fall 2022, 22(3) 988 Discussion Study results reflect the value of the expanded training provided through the project. Across almost all measures the training group indicated significant gains compared to the control group. These findings resulting from comparing the training group to a group that did not receive the training add to the literature that frequently lacks a control group (Acquavita et al., 2020; Benton et al., 2020). While it would be expected for participants of an enhanced training project to report higher levels of training than control group members, the finding of relevance is important. It reflects that training participants see the worth of their increased knowledge and skills specific to the competencies. Of particular value to integrated care training projects is the increased knowledge and skills of training participants in cultural competency, trauma-informed care, and communication. Research has noted these areas of training to be key for social workers entering interprofessional health settings (Mattison et al., 2017; Padykula et al., 2020). The one exception to significant findings was the Physical Environment subscale for the CC assessment. It is not surprising that this subscale would not see a significant improvement since all participants were students with little to no control of the environment of field placement. A potential impact to the project was the onset of the COVID-19 pandemic. Although adjustments needed to be made to the delivery of content and administration of pre- and post-test surveys, participant outcomes in these affected semesters were still positive. Limitations The surveys used in the study were self-report, thus a social desirability or self-serving bias could have influenced participants’ responses in a desire to appear proficient in relevant competencies. Still there was a significant difference in outcomes compared to the control group, reflecting that perceptions of proficiency and values were not as strong without participation in the project. As well, the study was conducted with one project at one public university in the southwest. Therefore, the data may not be generalizable to other stipend training programs. Additionally, the training sites where students participating in the project completed their field placements likely varied in the scope of training it provided to students. For example, all participants were not placed in hospitals. Integrated care occurs on a spectrum, so learning is not always the same or similar between individuals when they are in the field. Students also work with varied populations, each with their own unique needs addressed through integrated care, meaning that students typically receive specialized training relevant to their populations. And finally, the required and sudden shifting of practices at each field placement due to COVID-19 also influenced the training participants experienced. These variations in training may explain differing levels of knowledge and a focus on different competencies. Implications for Social Work Education There are benefits to providing monetary support for an educational program that prepares graduates to work with vulnerable populations. The ability of students to focus Benton et al./INCREASING SW STAKE IN IPE 989 more intently on the final field experience, including class and academic assignments along with the full-time internship, is supported through the provision of the stipend. Additionally, the number of hours worked in outside employment has been shown to negatively affects students’ GPAs, motivation levels, and retention rates (Tessema et al., 2014). Students who receive stipends to support their final field experience may be less likely to seek or continue employment while completing internship hours. Additionally, there is an ethical imperative to educate and train social work students as much as possible prior to their entrance to the field of social work. A core value for social work is competence (National Association of Social Workers [NASW], 2021). This study supports the literature emphasizing the need to prepare social workers for integrated behavioral healthcare (Fraher et al., 2018; Horevitz & Manoleas, 2013; Walsh et al., 2022) and increases the participants’ competence specifically to work in these settings. A recommendation for social work educators and administrators is to explore and advocate for more funding to provide enhanced integrated care training projects. Research has indicated barriers for integration of social workers into interprofessional settings (Ashcroft et al., 2018; Tucker & Webber, 2020). Specialized, stipend-supported training, such as those funded by HRSA, can support participant focus on necessary skills and the ability to educate others on what social work brings to the setting. Implications for Social Work Research One suggestion for increasing the research knowledge base is to track training participants’ employment after graduation. Given that the BHWET project does not include a contractual work obligation, it would be relevant to understand how many participants continued their employment within integrated care settings. Because the current study relied on self-reported evaluations of knowledge, skills, and attitudes, future research could also include the evaluations of participants’ supervisors and clients in order to triangulate the findings. Longitudinal research also could examine the extent to which participants in a training program such as BHWET are more likely to seek out employment in certain types of practice setting or have an increased likelihood of taking on leadership or supervisory roles. Another recommendation for further research is to explore to what extent graduates transfer their enhanced knowledge to the workplace. It has been estimated that only 10- 20% of knowledge gained in trainings is transmitted into practice behaviors (Austin et al., 2006). The potential long-term impact of enhanced training programs could be evaluated through a longitudinal approach. This approach would provide information as to whether the training received has been helpful in their social work roles and responsibilities. A further component of effectiveness to track longitudinally would be to specifically explore transfer and sustainability of skills and attitudes regarding working with diverse, underserved, or disadvantaged communities. This focus could provide insight to what works and where project improvements should be made for the maximum impact to integrated care settings and the populations they serve. ADVANCES IN SOCIAL WORK, Fall 2022, 22(3) 990 References Acquavita, S. P., Harley, D., Lee, B. R., Sacco, P., Levy, M., & Holmes, C. (2020). Preparing master of social work students for interprofessional practice. Journal of Evidence-Based Social Work, 17(5), 611-623. https://doi.org/10.1080/26408066.2020.1781730 Ashcroft, R., McMillan, C., Ambrose-Miller, W., McKee, R., & Brown, J. B. (2018). The emerging role of social work in primary health care: A survey of social workers in Ontario family health teams. Health & Social Work, 43(2), 109-117. https://doi.org/10.1093/hsw/hly003 Austin, M., Weisner, S., Schrandt, E., Glezos-Bell, S., & Murtaza, N. (2006). Exploring the transfer of learning from an executive development program for human services managers. Administration in Social Work, 30(2), 71- 90. https://doi.org/10.1300/J147v30n02_06 Bagdasaryan, S. (2012). Social work education and Title IV-E program participation as predictors of entry-level knowledge among public child welfare workers. Children & Youth Services Review, 34, 1590-1597. https://doi.org/10.1016/j.childyouth.2012.04.013 Benton, A., Feinman, E., Gonzalez, E., Rosen, A., & Anderson, L. (2020). Outcomes of a bundled intervention approach: Analyzing an enhanced MSW training project for integrated-care settings. Journal of Social Work Education, 58(2), 278-290. https://doi.org/10.1080/10437797.2020.1841698 Benton, A., & Iglesias, M. (2018). “I was prepared for the worst I guess”: Stayers’ and leavers’ perceptions of their Title IV-E education. Journal of Public Child Welfare, 12(3), 264-280. https://doi.org/10.1080/15548732.2018.1446067 Craig, S. L., Eaton, A. D., Belitzky, M., Kates, L. E., Dimitropoulos, G., & Tobin, J. (2020). Empowering the team: A social work model of interprofessional collaboration in hospitals. Journal of Interprofessional Education & Practice, 19, 1-9. https://doi.org/10.1016/j.xjep.2020.100327 Davis, T. S., Guada, J., Reno, R., Peck, A., Evans, S., Sigal, L. M., & Swenson, S. (2015). Integrated and culturally relevant care: A model to prepare social workers for primary care behavioral health practice. Social Work in Health Care, 54(10), 909-938. https://doi.org/10.1080/00981389.2015.1062456 Faul, F., Erdfelder, E., Lang, A., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191. https://doi.org/10.3758/BF03193146 Foster, G., Morris, M. B., & Sirojudin, S. (2013). Mental health workforce change through social work education: A California case study. Journal of Social Work Education, 49(3), 490-505. https://doi.org/10.1080/10437797.2013.796771 Fraher, E. P., Richman, E. L., Zerden, L. de S., & Lombardi, B. (2018). Social work student and practitioner roles in integrated care settings. American Journal of Preventive Medicine, 54(6), S281-S289. https://doi.org/10.1016/j.amepre.2018.01.046 https://doi.org/10.1080/26408066.2020.1781730 https://doi.org/10.1093/hsw/hly003 https://doi.org/10.1300/J147v30n02_06 https://doi.org/10.1016/j.childyouth.2012.04.013 https://doi.org/10.1080/10437797.2020.1841698 https://doi.org/10.1080/15548732.2018.1446067 https://doi.org/10.1016/j.xjep.2020.100327 https://doi.org/10.1080/00981389.2015.1062456 https://doi.org/10.1080/10437797.2013.796771 https://doi.org/10.1016/j.amepre.2018.01.046 Benton et al./INCREASING SW STAKE IN IPE 991 Gansle, K. A., & Ellett, A. J. (2002). Child welfare knowledge transmission, practitioner retention, and university-community impact: A study of Title IV-E child welfare training. Journal of Health & Social Policy, 15(3-4), 69-88. https://doi.org/10.1300/J045v15n03_06 Goode, T. (1999). Promoting cultural and linguistic competency: Self- assessment checklist for personnel providing primary health care services. Georgetown University Center for Child and Human Development. https://nccc.georgetown.edu/documents/Checklist%20PHC.pdf Horevitz, E., & Manoleas, P. (2013). Professional competencies and training needs of professional social workers in integrated behavioral health in primary care. Social Work in Health Care, 52(8), 752-787. https://doi.org/10.1080/00981389.2013.791362 IBM Corp. (2011). IBM SPSS Statistics for Windows, Version 20.0. Author. Jones, B., & Phillips, F. (2016). Social work and interprofessional education in health care: A call for continued leadership. Journal of Social Work Education, 52(1), 18-29. https://doi.org/10.1080/10437797.2016.1112629 Leake, R., de Guzman, A., Rienks, S., Archer, G., & Potter, C. (2015). NCWWI Traineeships: A national cross-site evaluation of child welfare stipend programs for ethnically diverse students. Journal of Social Work Education, 51, S299-S316. https://doi.org/10.1080/10437797.2015.1072419 Mattison, D., Weaver, A., Zebrack, B., Fischer, D., & Dubin, L. (2017). Educating social workers for practice in integrated health care: A model implemented in a graduate social work program. Journal of Social Work Education, 53, S72-S86. https://doi.org/10.1080/10437797.2017.1288594 National Association of Social Workers. (2021). NASW code of ethics. https://www.socialworkers.org/About/Ethics/Code-of-Ethics/Code-of-Ethics-English National Collaborative on Workforce and Disability for Youth. (2004). Knowledge, skills, and abilities of youth service practitioners: The centerpiece of a successful workforce development system. https://files.eric.ed.gov/fulltext/ED485705.pdf O’Donnell, J., & Kirkner, S. (2009). Title IV-E programs: Preparing MSW students for public child welfare practice. Journal of Teaching in Social Work, 29(3), 241-257. https://doi.org/10.1080/08841230903022050 Padykula, N., Berrett-Abebe, J., & Haven, T. (2020). Helping community partners build capacity within integrated behavioral health: A call to action for social work education. Advances in Social Work, 20(2), 266-282. https://doi.org/10.18060/23667 Putney, J. M., Sankar, S., Harriman, K. K., McManama O’Brien, K. H., Robinson, D. S., & Hecker, S. (2017). An innovative behavioral health workforce initiative: Keeping pace with an emerging model of care. Journal of Social Work Education, 53, S5-S16. https://doi.org/10.1080/10437797.2017.1326329 https://doi.org/10.1300/J045v15n03_06 https://nccc.georgetown.edu/documents/Checklist%20PHC.pdf https://doi.org/10.1080/00981389.2013.791362 https://doi.org/10.1080/10437797.2016.1112629 https://doi.org/10.1080/10437797.2015.1072419 https://doi.org/10.1080/10437797.2017.1288594 https://www.socialworkers.org/About/Ethics/Code-of-Ethics/Code-of-Ethics-English https://files.eric.ed.gov/fulltext/ED485705.pdf https://doi.org/10.1080/08841230903022050 https://doi.org/10.18060/23667 https://doi.org/10.1080/10437797.2017.1326329 ADVANCES IN SOCIAL WORK, Fall 2022, 22(3) 992 Rubin, M., Zerden, L., & McGuire, L. (2020). Leveraging the power of social work through interprofessional education. Advances in Social Work, 20(2), i-x. https://doi.org/10.18060/24406 Sampson, M. (2017). Teaching note—Meeting the demand for behavioral health clinicians: innovative training through the GLOBE Project. Journal of Social Work Education, 53(4), 744-750. https://doi.org/10.18060/24406 Smith-Osborne, A., & Daniel, K. (2017). Concurrent infusion of integrated behavioral health practice into social work field and classroom instruction. Journal of Social Work Education, 53, S17-S26. https://doi.org/10.1080/10437797.2017.1290564 Stanhope, V., Videka, L., Thorning, H., & McKay, M. (2015). Moving toward integrated health: An opportunity for social work. Social Work in Health Care, 54(5), 383-407. https://doi.org/10.1080/00981389.2015.1025122 Tessema, M. T., Ready, K., & Astanie, M. (2014). Does part-time job affect college students’ satisfaction and academic performance (GPA)? The case of a mid-sized public university. International Journal of Business Administration, 5(2), 1-10. https://doi.org/10.5430/ijba.v5n2p50 Tucker, L., & Webber, M. (2020). ‘Maybe a maverick, maybe a parent, but definitely not an honorary nurse’: Social worker perspectives on the role and nature of social work in mental health care. British Journal of Social Work, 51(2), 1-19. https://doi.org/10.1093/bjsw/bcaa202 Walsh, M., Jaggers, J., Satre, C., & Hall, J. (2022). Evaluation of interprofessional practice education for master of social work students: Student perspectives of a short-term IPE observation. Advances in Social Work, 22 (1), 233-250. https://doi.org/10.18060/25564 Zerden, L., Jones, A., Brigham, R., Kanfer, M., & Zomoro, M. (2017). Infusing integrated behavioral health in an MSW program: Curricula, field, and interprofessional educational activities. Journal of Social Work Education, 53, S59-S71. https://doi.org/10.1080/10437797.2017.1288595 Zerden, L., Jones, A., Day, S., & Lombardi, B. M. (2020a). Interprofessional collaboration: An evaluation of social work students’ skills and experiences in integrated health care. Journal of Social Work Education. https://doi.org/10.1080/10437797.2020.1743219 Zerden, L. D., Kanfer, M., & Levine, A. (2020b). Integrated field and collaborative opportunities for MSW students at campus health. Field Educator, 10(1), 1-10. https://fieldeducator.simmons.edu/article/integrated-field-and-collaborative- opportunities-for-msw-students-at-campus-health/ Zerden, L., Lombardi, B. M., Fraser, M. W., Jones, A., & Rico, Y. G. (2018). Social work: Integral to interprofessional education and integrated practice. Journal of Interprofessional Education & Practice, 10, 67-75. https://doi.org/10.1016/j.xjep.2017.12.011 Author note: Address correspondence to Amy Benton, School of Social Work, Texas State University, San Marcos, TX 78666. Email: ab68@txstate.edu https://doi.org/10.18060/24406 https://doi.org/10.18060/24406 https://doi.org/10.1080/10437797.2017.1290564 https://doi.org/10.1080/00981389.2015.1025122 https://doi.org/10.5430/ijba.v5n2p50 https://doi.org/10.1093/bjsw/bcaa202 https://doi.org/10.18060/25564 https://doi.org/10.1080/10437797.2017.1288595 https://doi.org/10.1080/10437797.2020.1743219 https://fieldeducator.simmons.edu/article/integrated-field-and-collaborative-opportunities-for-msw-students-at-campus-health/ https://fieldeducator.simmons.edu/article/integrated-field-and-collaborative-opportunities-for-msw-students-at-campus-health/ https://doi.org/10.1016/j.xjep.2017.12.011 mailto:ab68@txstate.edu