This work is licensed under a Creative Commons Attribution 4.0 International License. The Impact of a Title IV-E Program on Perceived Practice Skills for Child Welfare Students: A Review of Five MSW Cohorts Elizabeth Jane Greeno Lisa Fedina Berenice Rushovich Caroline Burry Debra Linsenmeyer Christopher Wirt Abstract: Title IV-E Education for Public Child Welfare training programs are designed to build knowledge and practice skills among students and current child welfare workers in efforts to build a competent and highly trained workforce. A mixed methods study was conducted to: 1) measure changes in MSW Title IV-E students' perceived confidence to perform skills across 13 practice content areas for public child welfare practice, and 2) to explore students’ perceptions of their competency for child welfare practice. This study also focused on the impact of prior child welfare experiences on perceived child welfare knowledge and skills among Title IV-E students. A total of 224 Title IV-E MSW students over the course of five academic cohorts participated in this study. Surveys were conducted at three time points: pretest, posttest, and retrospective pretest. Twenty focus groups were conducted during the study time period. Findings indicate gains across all practice content areas with the largest gains in areas of working with the courts and conducting assessments. Qualitative findings assessing student's perception of competency to practice in child welfare include themes of students’ preparation to practice post-graduation and differences between the students’ experiences in the IV-E program and what they witness in the field. Specific practice area recommendations include addressing workers’ age and prior experience in Title IV-E seminars and trainings as well the importance of Title IV-E field instructors in helping to prepare students for child welfare practice. Keywords: Title IV-E; child welfare knowledge; child welfare practice; child welfare training Over the last 20 years, federal funding through Title IV-E and Title IV-B, Section 426, (Adoption Assistance and Child Welfare Act, P.L. 96-272) has provided states with resources to design, implement, and facilitate child welfare training programs for future and current public child welfare employees in the United States. Workforce retention and specialized child welfare knowledge, skills, and competencies are the key outcome objectives of these training programs. A considerable amount of research has been dedicated to evaluating the impact of Title IV-E training on workforce retention, suggesting that IV-E programs are a significant predictor of stronger workforce retention outcomes and therefore are a valuable tool in reducing staff turnover in public child welfare (Zlotnik, _________________ Elizabeth J. Greeno, MSW, PhD is a Research Associate Professor at the University of Maryland School of Social Work (UM SSW) in Baltimore, MD. Lisa Fedina is a PhD Candidate at UM SSW. Berenice Rushovich, MSW is a Research Scientist with Child Trends in Bethesda, MD. Caroline Burry, PhD is an Associate Professor at UM SSW. Debra Linsenmeyer, MSW is the Educational Director of the Title IV-E program at UM SSW. Christopher Wirt, MSW is a Clinical Instructor at UM SSW. Copyright © 2017 Authors, Vol. 18 No. 2 (Fall 2017), 474-489, DOI: 10.18060/21058 https://creativecommons.org/licenses/by/4.0/ ADVANCES IN SOCIAL WORK, Fall 2017, 18(2) 475 DePanfilis, Daining, & McDermott Lane, 2005). Additionally, Title IV-E programs place a strong emphasis on building and strengthening knowledge and practice skills among program participants and subsequently, within the child welfare workforce. In 2008, the Council on Social Work Education (CSWE) adopted a competency-based education framework and described social work competence as “the ability to integrate and apply social work knowledge, values, and skills to practice situations in a purposeful, intentional, and professional manner to promote human and community well-being” (CSWE, 2015, p. 6). CSWE (2015) defines values as recognizing social, economic, cultural, and political climates and the impact of these areas on the clients they may serve. Indeed, the inclusion of values in definitions of competence sets the social work profession apart from other professions that do not consider values to be an integral part of competence in addition to knowledge and skills. This is especially salient for assessing competencies in child welfare, as not all child welfare workers have professional degrees in social work and thus agencies often provide training on social work values (Barth, Lloyd, Christ, Chapman, & Dickinson, 2008). Child Welfare Knowledge and Skill Acquisition Overall, research indicates that Title IV-E participants score higher on public child welfare measures of knowledge and practice skills compared to non-program participants, suggesting that IV-E education provides students with fundamental child welfare knowledge and practice skills (Bagdasaryan, 2012; Fox, Miller, & Barbee, 2003; Franke, Bagdasaryan, & Furman, 2009; Jones & Okamura, 2000; Yankeelov, Barbee, Sullivan, & Antle, 2009). Hartinger-Saunders and Lyons (2013) identified 10 peer-reviewed studies that used empirical data to evaluate Title IV-E programs. Only 4 of the 10 studies measured competency through knowledge and skill assessment among IV-E participants, while the remaining studies focused on areas of job satisfaction and retention. Based on knowledge tests, 3 of the 4 studies indicated improved knowledge gains among Title IV-E participants (see Franke et al., 2009; Gansle & Ellet, 2002; Jones & Okamura, 2000). One study assessed findings from an evaluation of a new employee pre-service training program and revealed that newly-hired Title IV-E employees scored higher on both pre and posttest measures of child welfare content knowledge (i.e., areas such as permanency planning, maltreatment definition) and knowledge associated with application of practice skills (i.e., vignettes for decision-making related to child welfare cases) compared to newly hired non- Title IV-E employees (Franke et al., 2009). Previous Child Welfare Experience A variety of external factors may influence student practice skills in child welfare. Specifically, one overlooked area is the influence of prior child welfare experience. Child welfare experience prior to a formal social work education often includes volunteer experience or paraprofessional roles that may introduce students to professional child welfare practice and build skills specific to child welfare settings. This experience may also include generalist practice courses or exposure as well as a values orientation to the field. Previous child welfare experience exposes and socializes students to the [social work] field, which subsequently increases self-perceived competency in child welfare practice (Cheung Greeno et al./PERCEPTIONS OF RECIDIVISM 476 & Tang, 2010). Very few studies, however, have explored the impact of prior child welfare experience on practice skills and knowledge among IV-E participants. Overall, the impact of socialization to the field on child welfare competency, social work values attainment, and practice skills, requires further study. University of Maryland, Baltimore Title IV-E Program The Title IV-E Public Child Welfare Education Program in the State of Maryland is a collaborative effort between the University of Maryland, Baltimore School of Social Work (UMB SSW) and the Maryland Department of Human Resources. The mission of the Title IV-E program in this state is to prepare Master of Social Work (MSW) students for social work practice to provide family-focused and strengths-based public child welfare services to families and children. The program is funded through federal Title IV-E funds for state and local child welfare training. The primary objectives of the program are to increase the number of professionally trained social workers in Maryland’s public child welfare system and to further the development of core values, knowledge, and skills necessary for competent provision of services to children and families served in public child welfare. At the MSW level, UMB SSW’s Title IV-E Program recruits students who are interested in the Families and Children specialization. The Families and Children specialization prepares graduates for clinical, community development, and human services management practice with families and children. It also provides focused studies in family and child programming and policies, with particular emphasis on public child welfare issues. Students in the IV-E program are required to complete specialized coursework that includes a child welfare practice class and training in child welfare. Trainings include monthly seminars on child welfare topics and specialized trainings, such as court and testimony preparation. Title IV-E students also fulfill field placement requirements at local public child welfare agencies. The student units at UMB SSW are composed of a combination of MSW foundation and MSW advanced clinical students under the supervision of a university field instructor who is assigned to the agency in which the unit is located. The field instructors provide individual instruction to each student in the unit and conduct case presentations and seminars relevant to practice issues. Students are assigned child welfare cases being served in the local child welfare agency and work as a team with child welfare workers to provide services to children and families. Purpose of Study The current study has two primary aims: first, to measure changes in MSW students’ perceived confidence to perform skills across 13 public child welfare practice content areas before and after participation in the university's Title IV-E Education for Public Child Welfare program and second, to explore students’ perceptions of their competency to practice in child welfare. The following research questions were addressed in the current study: 1) Is participation in the Title IV-E program associated with changes in student self- reported confidence across 13 public child welfare practice content areas? 2) Is there a difference in student self-reported confidence at pretest and retrospective pretest? 3) Are confidence gains influenced by student demographic characteristics? 4) Is prior child ADVANCES IN SOCIAL WORK, Fall 2017, 18(2) 477 welfare experience associated with higher levels of perceived confidence prior to and following participation in the Title IV-E program and retrospectively? and 5) How do students perceive their competency and readiness for their post-graduation, full-time employment with a public child welfare agency? Questions 1 through 4 are addressed quantitatively and question five is addressed through focus groups. The current study strengthens the empirical body of research on Title IV-E programs by examining perceived practice skill confidence across five MSW cohorts. Additionally, two critical, yet understudied, areas within Title IV-E research are addressed. First, the impact of prior child welfare work experience on perceived practice skills among IV-E participants is evaluated. Examining the effect of prior child welfare experience on IV-E participants’ perceptions of their skills may highlight specific training needs for IV-E students with prior experience. Second, few studies have assessed students’ perceptions of their competency and readiness to begin a full-time position with a public child welfare agency post-graduation. Assessing students’ perceptions of their competencies and ability to practice after completing the Title IV-E program can provide insight into possible training needs. In addition, specific strategies that helped students gain a sense of competence to practice are presented. Methods Both qualitative and quantitative methods were used to evaluate the university’s Title IV-E program. Pretest, posttest, and retrospective pretest surveys were administered and focus group interviews with IV-E participants were conducted to analyze students’ perceived practice competency. Qualitative and quantitative methods held equal status in the research design and both types of data were collected simultaneously throughout the study (see Johnson & Onwuegbuzie, 2004). Additionally, the use of the retrospective pretest evaluation approach has been shown to be an effective way of accounting for response shift bias that would otherwise not be accounted for in traditional pre-posttest designs (Pelfrey & Pelfrey, 2009). Using retrospective pretest-posttest design mitigates the underestimation of program effects (Pratt, McGuigan, & Katzev, 2000) that can occur in curricula such as Title IV-E training; that is, as an understanding of social work and child welfare concepts grows, respondents’ original understanding of their prior knowledge may change. Sampling and Recruitment Recruitment for study participants occurred through a two-stage process. First, program staff provided all Title IV-E participants with information on the study during IV- E program orientation, which is held annually in the beginning of each academic year. All IV-E students were invited to participate in the evaluation. Participants were also provided with Title IV-E evaluation goals, objectives, and consenting procedures. Second, evaluation consent forms and stamped, self-addressed envelopes were mailed to all students who agreed to participate in the evaluation. This study is comprised of students from five cohorts who participated in the Title IV-E program during the academic years of 2009 to 2013. Over this time period, 256 students were eligible to participate in the evaluation. Among the 256 eligible students, a total of 224 students participated, yielding Greeno et al./PERCEPTIONS OF RECIDIVISM 478 a response rate of 88%. University Institutional Review Board (IRB) approval was obtained for the current study. Data Collection Procedures Participants completed a Self-Assessment Child Welfare Competency Practice Skills survey at the beginning (pretest) and end of the academic year (posttest) in which they participated in the Title IV-E Program. At the end of each academic year, students were asked to rate themselves on how confident they believed they were now (standard posttest) and how confident they thought they were at the beginning of the year (retrospective pretest measure). This measure asked participants to rate, on a 10-point scale, their level of confidence to perform 13 practice skills related to public child welfare practice (see Table 3 below for a list of all 13 practice skills). The Self-Assessment Child Welfare Competency Practice Skill Survey was created using the knowledge and skill competencies derived from the key competencies identified for this IV-E program (for more information on key competencies, see Zlotnik, 1997) across 13-different child welfare content skill areas. Skills such as engaging families, developing a service plan, and working with the courts are assessed. Scores of 0-4 are considered to be perceived low confidence in ability to practice the skill; scores of 5-7 are considered to be perceived levels of moderate confidence in ability; and scores of 8-10 are considered to be perceived levels of excellent confidence in ability. All surveys were administered through an online survey software program. At the pretest measure (beginning of the academic year), students were also asked to answer a demographic survey that captured age, race and ethnicity, gender, and prior child welfare experience. For prior child welfare experience, students were asked to indicate their years of experience and to describe by a text write-in option their prior experience. Prior child welfare experience was counted as prior experience if it included working in any child welfare setting (i.e., public child welfare agency, private foster care settings, group homes or residential treatment centers), for non-profit agencies involved in child welfare work or working in hospital or mental health settings. Following recommendations from Dillman, Smyth, and Christian (2008), a web-based survey and emailing strategy were used. Specifically, students were emailed an initial survey invitation, which included a link to the online survey. Over the course of three weeks, students were sent three reminder emails to complete the survey. Completed survey data were available for all 224 participating students. However, demographic data were not available for the full sample (data available for n=146). All quantitative data were analyzed using SPSS Version 22.0. Paired samples t-tests, one-way ANOVAs, and multiple regression analyses were conducted to assess survey outcomes. A total of 20 focus group interviews (approximately 1 hour each) were conducted by the lead IV-E evaluator (first author). The groups were conducted at the end of each academic year at field instruction sites. A semi-structured focus group questionnaire was developed to assess students’ perceived competency in child welfare, as well as their perceived readiness and ability to apply newly learned skills to their practice in child welfare. Students were specifically asked, "how competent do you feel about your ability ADVANCES IN SOCIAL WORK, Fall 2017, 18(2) 479 to practice as a public child welfare worker post graduation” (prompt: “How do you feel about your ability to competently practice as a new child welfare worker?”) Among the 224 students participating in the evaluation, a total of 155 students participated in focus groups, yielding a participation rate of 69% for the qualitative portion of the evaluation. Narrative data were analyzed using the qualitative data analysis software program NVivo (QSR International Pty Ltd, 2010). Qualitative analyses were conducted through a four- step process. First, all focus group interviews were audio-recorded and transcribed verbatim. Second, interviews were analyzed using open coding techniques and recurrent themes were identified. Third, a constant comparative method, following Padgett’s (2004) qualitative analysis framework, was employed to compare themes that emerged from the data. Finally, categories and themes were organized and interpreted (Krueger, 1988; Krueger & Casey, 2009). Results Demographics Demographic data for age and ethnicity were available for 3 out of 5 academic years (n=146). Table 1 presents ethnicity data for participants. Study participants were primarily female (92%), White/Caucasian (39%) or Black/African American (20%) (See Table 1). The average age of participants was 27.6 years (range 21-55). There was approximately a two-year age difference between those who indicated they had prior child welfare experience (M=29) compared to those without prior child welfare experience (M=27). However, this was not a statistically significant difference, F=(1, 111)=2.972, p=.087. See Table 2 for details. The length of prior child welfare experience was available for only 64 participants; the average number of years of prior experience was 3.3 (SD=4, range 6 months – 22 years). Table 1. Ethnicity of MSW Student Participants (n=146) Ethnicity n (%) White/Caucasian 87 (39%) Black/African American 45 (20%) More than one race 8 (4%) Hispanic 6 (3%) Table 2. Age Data for IV-E Students With and Without Child Welfare Experience (n=146) Mean Mode Median SD Range Age-All 27.6 24 25 6.6 21-55 Students with prior child welfare experience (n=64) 29 26 25 7 21-55 Students without prior child welfare experience (n=82) 27 23 24 5 22-46 Changes in Perceived Practice Skills after Program Participation Findings for pre and post-practice skill assessments are provided in Table 3. Overall, students demonstrated increased self-reported confidence across all 13 skills. Students Greeno et al./PERCEPTIONS OF RECIDIVISM 480 indicated perceived levels of moderate gains across all 13 practice skills at pretest. Scores ranged from 1 to 2 points higher between pre and posttest, suggesting that students had increased self-perceived child welfare practice skills after completing Title IV-E education. By posttest students were classified as either having perceived levels of moderate to excellent practice skills. Students demonstrated the largest gains between pre-post test scores within the areas of working with court systems and conducting assessments for child sexual abuse. Students had modest gains in areas such as diversity, use of self in practice, working in settings other than their current field placements (e.g., investigations, foster care), and working with a variety of child developmental stages (e.g., infancy, adolescence). Table 3. Pre and Post-Program Participation Scores on Self-Assessed Competencies (n=224) Child Welfare Practice Content Area M Pretest M Posttest M Difference 95% CI t p LL UL 1. Engaging families in assessment 6.8 8.2 1.4 0.81 1.68 5.68 <.001 2. Developing a service plan 6.7 8.1 1.3 0.89 1.77 6.00 <.001 3. Collaborating as a team member 6.9 8.4 1.5 0.88 1.76 5.96 <.001 4. Diversity & use of self 7.4 8.4 1.0 0.56 1.25 5.18 <.001 5. Assessing for abuse & neglect 6.7 8.2 1.5 0.97 1.87 6.21 <.001 6. Using clinical skills 5.9 7.6 1.7 1.21 2.12 6.95 <.001 7. Working in other settings 7.4 8.4 1.0 0.49 1.56 3.82 <.001 8. Assessing & intervening for substance abuse 6.0 7.4 1.4 0.83 1.80 5.36 <.001 9. Assessing & intervening for sexual abuse 4.9 6.9 2.0 1.32 2.48 6.52 <.001 10. Arranging out-of-home placement 5.2 6.9 1.7 0.57 1.71 4.03 <.001 11. Permanency planning 6.1 7.2 1.1 0.42 1.57 3.43 0.01 12. Working with the courts 5.2 7.1 1.9 1.14 2.44 5.47 <.001 13. Working with developmental stages 7.3 8.3 1.0 0.69 1.42 5.74 <.001 Notes. CI=confidence interval; LL=lower limit; UL=upper limit Retrospective Pretest Findings Table 4 details findings for the differences between the pretest and the retrospective pretest. Ten of the thirteen skill areas were statistically significant, indicating differences in student perception from pretest to retrospective pretest. Students did not report gains in the areas of using clinical skills, assessing for sexual abuse, and working with court systems. Overall, at the beginning of the academic year, students reported perceptions of a higher level of confidence for practice skills than they did when retrospectively assessing their skill level. Demographic Differences The demographic variables of age and ethnicity were used to assess differences at the three survey time points. Results from the multiple regression analysis suggest significant differences at pretest. Specifically, the demographic variables had an impact on the participant's overall score (R²=.094, F(2,110)=.5.707, p=.004). Ethnicity was not a significant variable in differences at pretest. However, age was significant. For each ADVANCES IN SOCIAL WORK, Fall 2017, 18(2) 481 additional year in age, participants scored 1 point higher on the practice skill assessment (p=.002) at pretest. Multiple regression analysis was also used to assess demographic differences at retrospective pretest (R²=.094, F(2,106)=1.602, p=.206) and posttest (R²=.030, F(2,109)=1.671, p=.193). Both models were non-significant; demographic variables did not have a significant difference on the retrospective pretest or the posttest scores. Table 4. Pretest and Retrospective Pretest Scores on Self-Assessed Competencies (n=224) Child Welfare Practice Content Area M Pretest M Posttest M Difference 95% CI t p LL UL 1. Engaging families in assessment 5.9 6.8 0.9 .35 1.36 3.36 <.001 2. Developing a service plan 5.9 6.7 0.8 .26 1.31 2.98 .004 3. Collaborating as a team member 6.3 6.9 0.6 .26 1.21 3.07 .003 4. Diversity & use of self 6.5 7.4 0.9 .46 1.33 4.09 <.001 5. Assessing for abuse & neglect 6.1 6.7 0.6 .18 1.23 2.67 .009 6. Using clinical skills 5.4 5.9 0.5 -.05 1.07 1.79 .076 7. Working in other settings 6.2 7.4 0.2 .65 1.75 4.30 <.001 8. Assessing & intervening for substance abuse 5.4 6.0 0.6 .05 1.20 2.16 .033 9. Assessing & intervening for sexual abuse 4.9 4.9 -- -.57 0.66 0.16 .876 10. Arranging out-of-home placement 4.8 5.2 0.4 .30 1.50 2.98 .004 11. Permanency planning 5.0 6.1 0.1 .59 1.79 3.90 <.001 12. Working with the courts 4.8 5.2 0.4 -.22 1.05 1.28 .202 13. Working with developmental stages 6.6 7.3 0.7 .28 1.15 3.24 .002 Notes. CI=confidence interval; LL=lower limit; UL=upper limit Prior Child Welfare Experience Data were available for 164 students regarding their prior (to the start of the academic year) public child welfare experience. Table 5 presents results from three one-way ANOVA analyses that explored: 1) the difference between students with and without prior public child welfare experience before and after program participation, 2) differences on the retrospective pretest, and 3) overall change in total practice skill score. Students with prior public child welfare experience had significantly higher scores on the self-assessment measure prior to program participation (i.e., pretest). However, there were no significant differences at posttest and retrospective pretest time points (see Table 5). Table 5. Differences Between Total Competency Scores for Students With and Without Prior Public Child Welfare Experience Time point Experience n M Score F p Pre-IV-E None 68 5.6 3.75 <.001 Prior 62 7.0 Post IV-E None 49 7.8 0.59 0.59 Prior 51 7.8 Retrospective None 49 5.5 1.16 0.284 Prior 52 5.9 Greeno et al./PERCEPTIONS OF RECIDIVISM 482 Qualitative Findings Students were asked to assess their perceived child welfare competency gains throughout their graduate education in conjunction with their Title IV-E program experience and to gauge their perceived or anticipated competency post-graduation as a projected full-time employee in a public child welfare agency. Three main themes emerged from the qualitative data: 1) preparation to practice post-graduation; 2) difference between IV-E field experience and what they witness in practice during field; 3) work and life balance. Qualitative themes are presented below, along with supporting documentation from participants. Preparation to Practice Post-Graduation Overall, student responses indicated two distinct perceptions of competency readiness to practice post-graduation from the IV-E program: those who felt competent to practice post-graduation and those who felt less prepared to practice. Students who felt competent or ready for the "challenge" reported that they felt as prepared as they could be to practice in child welfare and appeared to anticipate the unexpected challenges they would likely face as an employee. These students understood that they would eventually be exposed to situations and experiences they had not previously encountered in their field placements while in the IV-E program. Still, this group of students felt confident that they possessed the foundation skills necessary to practice competent social work and manage unexpected challenges. These students also described strategies that increased their self-confidence in practicing social work and managing unexpected challenges. Two main strategies were repeatedly mentioned: field instructors and seminars. Students reported that they learned the most from their field instructors. Field instructors provided the opportunity to “hone skills,” process cases, and discuss child welfare policy and practice. They provided students with practical and hands-on experience and were able to model child welfare practice skills for students. Other descriptions of field instructors included “sounding boards” or people who eased the students’ anxiety. Seminars provided a means of structured and concrete teaching of child welfare practice and policy. The resources and discussions held during these seminars were seen as useful and applicable. ...You’re just never going to, you can’t prepare us for everything but I think you’ve got all the bases down.-Student, 2011 I've learned to gain confidence to be able to speak to people and confront them about the things that they’re resistant about,...and having the courage to do that. It definitely is a learning process to build that level of confidence.-Student, 2012 An additional strategy students indicated as valuable to achieving competency was shadowing other workers in the field. Shadowing experiences had two main benefits: 1) exposing students to numerous and varied situations/experiences in their units and 2) allowing students to see how other social workers conduct themselves and implement practice skills in the field. Other students thought that despite the resources, preparation, and training received from their public child welfare field placements and the IV-E program, they still did not ADVANCES IN SOCIAL WORK, Fall 2017, 18(2) 483 feel prepared to graduate and practice at a public child welfare agency. Students who felt less prepared also mentioned the role of shadowing in their field placement experiences. Unlike students who felt competent and confident, the shadowing experiences, which were often identical (i.e., multiple students shadowed the same child welfare workers), appeared to make this particular group of students feel more anxious about practicing in a public child welfare agency post-graduation. These students reported satisfaction with the IV-E program, but appeared to have the feeling they could never "get enough practice" before becoming full-time employees. These students wanted to feel prepared in all areas of child welfare so they could be "prepared for anything." The following quotes from two students illustrate this theme: Everything scares me....just scares me. I’m scared. I’m really not prepared. Not that this IV-E experience hasn’t been really awesome. And it has helped me out a lot. I think I’m, I mean I don’t know if I’ll ever feel prepared.-Student, 2014 .The way I see it, it’s really, this job is so much, seems so much on the job versus, ya know, reading this or class training...there’s things that you’re not going to know until you come across it…there’s tons I haven’t come across yet.-Student, 2013 Difference from IV-E Field Experience and What They Witness in Practice All students reported concern about their ability to transfer what they learned from the IV-E program to their practice. Students believed they were being trained to provide quality practice with their child welfare clients, though they reported witnessing current workers doing mainly case management activities and not having opportunities to practice clinical skills. Students viewed their role as a child welfare worker as combining both clinical skills and case management activities. Students reported that there is a strong emphasis on developing and mastering clinical skills and competence within the Title IV-E program. However, students’ view of workers in their field placement was that there was no emphasis on, or no time to use, clinical skills. Instead, the focus was to “get the work completed.” I mean, here [in field] we spend a lot of time being with the client, what the client wants, and following their lead...but the time constraints in a DSS work relationship, you don't have time to really explore and get them to elaborate and talk. You've got this checklist that's mandated.-Student, 2011 Some things are beyond your control because the caseloads are cumbersome, so, and like, documentation becomes a priority so you want to spend this extra hour there engaging but you want to get back because you know you've got this stuff to do. So it's a struggle.-Student, 2013 Students also reported on various aspects of professional behavior, including attitudes among child welfare workers at their field placements, which brought about additional concerns for students. Based on what they had witnessed from workers, students were concerned that once they began working they would become overwhelmed with job Greeno et al./PERCEPTIONS OF RECIDIVISM 484 demands and "lose" their ability to both implement comprehensive clinical child welfare practice skills and perform the necessary case management activities. Well what I've noticed is there's just like this tension where we learn the right way to do things a lot of times in the IV-E seminars and [in] the school and a lot of good practices and then you go out to field and it's a little discouraging sometimes when you are around a bunch of social workers that have become a little bit cynical and you have people telling you just do the bare minimum, like that's all they want you to do. And it's just like, like for me, I think that you're kind of like if I want to do this, I care about it, so you're not really supported in that. They kind of look [at you] like you're going to burn yourself out.-Student, 2013 Work and Life Balance Additionally, many students reported future concerns related to their ability to balance having a family and being able to practice the IV-E skills they had learned. Several students (the majority of participants in the evaluation were female) reported plans or hopes of having their own family within five years after graduation and reported concerns that they would not have the time and energy to have both a family and a job in public child welfare. Students appeared to be influenced by what they had witnessed from child welfare workers in the field placement and perceived there would be future conflict over the demands of the job, being overwhelmed, and obligations to their family. ...You could do this job with, if you really wanted, it would be hard. I think it’s good when you're young and you’re vibrant but I could see it getting, it being really hard to juggle this specific job in family welfare and also having a family. -Student, 2014 Discussion The multifaceted nature of social work practice in the field of public child welfare is undeniably challenging, especially for new social work practitioners. Rigorous and comprehensive training in child welfare practice, such as the preparation provided in Title IV-E programs, is needed to build an efficacious public child welfare workforce. Additionally, Jones and Okumura (2000) suggest that the more competent child welfare workers feel in their practice, the more likely they are to remain in public child welfare over time. Overall, findings from the current study provide support for UMB SSW's Title IV-E educational program. Results from participants’ pre-posttest scores indicate increased perceived confidence to practice in child welfare settings. This finding is consistent with previous studies on IV-E programs (Bagdasaryan, 2012; Franke et al., 2009; Gansle & Ellet, 2002; Jones & Okamura, 2000). Measuring perceived confidence in practice skill ability among IV-E participants is a step towards measuring actual competency in child welfare practice. This study provides empirical evidence through a mixed methods approach to support UMB SSW's Title IV-E program goals, which are focused on graduating social workers who are competent to practice in public child welfare. Still, there were differences in participants’ perceived skills for specific areas of child welfare practice. Pre-posttest results suggest only modest gains in areas of diversity and ADVANCES IN SOCIAL WORK, Fall 2017, 18(2) 485 use of self, working in other settings, and working with children of different developmental stages. However, for all three practice domains students entered scoring in the moderate range of perceived skills and there was approximately a one-point difference from pretest to posttest. This suggests students entered the MSW program with a fair amount of confidence in these areas, but were still able to make gains through their Title IV-E experience. Greater gains were made in areas of assessing and using clinical skills, arranging out-of-home placements, intervening for sexual abuse, and working with the courts. These gains would tend to support the use of some specific strategies employed by this IV-E program. For instance, as IV-E students have historically stated that they were very anxious about preparing for and testifying in court, this IV-E program includes a full- day mock court experience. Similarly, feedback from past students about their need for additional preparation for addressing child sexual abuse resulted in additional emphasis on this topic in their required child welfare practice course. Other IV-E programs, therefore, may want to consider adding or emphasizing training and educational support in these areas. Students reported differences between pretest and retrospective pretests in all but three areas; clinical skills, assessing for sexual abuse, and working with court systems. This finding is not unexpected as these areas require specific training and exposure for students to feel comfortable with the skill and it is highly likely that students did not have these skills at the start of the academic year (pretest) and thus the retrospective pretest would not have picked up any differences. For the other practice areas, students rated themselves higher at pretest but at retrospective pretest students likely had more perspective after having two semesters of field and classes, and thus retrospectively rated themselves lower in 10 of the 13 practice areas. Other findings from this study suggest several interesting and important patterns. For example, age and prior child welfare experience were associated with increased self- perceived confidence in practice skill ability at the beginning of the IV-E program. There may be an interaction between age and having prior child welfare workforce experience, as those who had prior child welfare workforce experience were on average two years older than those who did not have prior child welfare workforce experience; however, this was not a statistically significant difference. Additionally, at the conclusion of the academic year, no difference was found on the posttest or the retrospective pretest for those with or without prior child welfare experience. In addition, age was not a predictor of scores at posttest or retrospective pretest. This suggests that upon completion of the IV-E program, all students had fairly equal levels of competency. All students gained confidence after participation in IV-E, but those with prior experience in child welfare or who were older did not have additional or greater gains than those without prior experience. This finding is somewhat surprising, since other research suggests that prior work experience contributes to higher levels of perceived competency after training (Cheung & Tang, 2010). Explanations for this finding may be that students with prior experience truly thought they had more confidence in the practice area than those without experience and through their academic and field experience, they realized there were new skills they could learn or skills on which they could improve. However, it is likely that the rigor of the Title IV-E participation that included specialized field experiences and coursework provided younger students and students without prior experience the opportunity to learn or catch up to their counterparts. Regardless, Title IV-E programs may need to be tailored to provide Greeno et al./PERCEPTIONS OF RECIDIVISM 486 maximum benefit for students with prior child welfare experience and/or older students. Future research is needed to assess the differences in needs for students with and without prior child welfare experience as well the impact, if any, of the age of the student. Previous research suggests the possibility that older child welfare workers may benefit from a holistic approach to teaching and learning (Franke et al., 2009) and this may address different learning styles. Training for older child welfare workers and/or those returning to school after work experience may need to address how the older student assimilates new information and material with what they have already learned or have already been practicing. Tailoring trainings, supervision, and for this university, seminars to these specific subpopulations of Title IV-E students likely will impact perceived practice abilities. Findings from the qualitative portion of this mixed methods study also yield interesting themes related to preparedness and perceived competency. Although all students in the IV- E program receive the same program elements (e.g., field placement, field supervision, training seminars, and academic coursework), there were differences in reported preparedness to practice post-graduation. Several focus group participants reported that the IV-E program prepared them for practice. This “more prepared” group also reported being aware of and feeling prepared to address the unknown elements of public child welfare they would likely encounter post-graduation. Field instructors seemed to be invaluable to these students, providing them with learning opportunities, support, and guidance for child welfare practice. For the remaining focus group participants, they appeared to feel "less prepared" and it seems that their IV-E experience appeared to make them feel more anxious and even less prepared for child welfare practice post-graduation. This may reflect different student learning styles. In addition, focus groups were conducted at the end of the academic year and it is likely that these students either expressed or would have expressed anxiety (if asked) at other times during the academic year. These types of students may require additional supervision and support by their assigned field instructors. Preparation for the inevitability of “unexpected challenges” and uncertainty faced in child welfare may reduce anxiety among students. Perception of preparation to practice in child welfare is likely highly influenced by personal characteristics and future research should identify these specific characteristics. Overall, students’ reports of either feeling more prepared or less prepared suggest the need for IV-E programs and IV-E field instructors to prepare students for unexpected challenges in the field and to remain attuned to students’ individual education and training needs. Additional findings from the focus groups included students’ perceptions of practice competency post-graduation as full-time employees. Students noted discrepancies between what they were being taught in IV-E curricula and what was actually happening in the field. While students believed they were being taught means to be competent and current clinical skills to practice in their field units (in local departments of social services), they witnessed child welfare workers’ not having opportunities to perform these clinical skills. Students identified that issues of time management and agency-mandated task requirements likely explain workers’ inability to conduct clinical practice with children and families. Indeed, the transition from being a social work student to a public child welfare employee is difficult to navigate. This finding also suggests the need to prepare students for the ADVANCES IN SOCIAL WORK, Fall 2017, 18(2) 487 organizational climate of child welfare agencies. Students may likely benefit from post- graduation support from Title IV-E programs (e.g., access to seminars and trainings, IV-E alumni networking). Other possible avenues may include Title IV-E programs working with child welfare supervisors and administrators in efforts to provide clear communication regarding the skills the students and future employees are being taught and will be using in the field as a child welfare worker. Limitations There are several limitations to the current study. First, this study was conducted at one university using a convenience sample of Title IV-E students and findings may not be generalizable. Additionally, the use of self-report measures, including a retrospective measure, is susceptible to recall bias, which impacts the reliability of study findings. While quantitative measures distinguished between those students with and without prior child welfare experience, the questions in the focus groups and therefore subsequent analyses did not account for this difference. This is a significant limitation as those students with prior experience may have had different perspectives on preparation for practice and the types of practice they witnessed in the field. The quantitative measure in this study assessed perceived confidence of child welfare practice skills and did not assess other areas of social work competence, such as values. In addition, focus groups participants were asked about their perceived competency to practice. Questions used in the focus groups did not address student definitions of competency. Assessing student definitions of competency may have clarified whether or how students view competency in relation to CSWE definitions that include knowledge, skills, and values. In addition, given the number of t-tests completed for the analyses there is an inflated Type I error rate. Finally, IV-E students are placed in a variety of departments and units within the public child welfare system which may impact their perceived practice skills and was not accounted for in this study. Additionally, the amount of experience and expertise among field instructors supervising IV-E students may also impact student competencies. Differences in students’ assigned units, including the structure of the departments, the types of cases to which students were exposed, and the level of experience among field instructors, were not evaluated. Therefore, the extent to which these factors impacted perceived competency among students is unknown. Future studies might explore the various aspects of practice settings and their moderating or meditating effects on perceived competency among IV-E students. Conclusion and Recommendations Funding for Title IV-E programs has provided universities and public child welfare agencies with substantial support to develop training programs that build and retain a competent child welfare workforce. Research suggests that graduates of IV-E programs have more on-the-job competence when compared to non-IV-E participants (Government Accountability Office, 2003). However, the particular components of IV-E education that contribute to competency gains remain unknown (Bagdasaryan, 2012). Additionally, no research has examined the impact of competency gained through IV-E programs on child and family outcomes (Hartinger-Saunders & Lyons, 2013). Findings from the current study suggest that Title IV-E programs be tailored to students with different learning styles and/or Greeno et al./PERCEPTIONS OF RECIDIVISM 488 for older students or those students with prior child welfare experience. Additionally, students need to be prepared for the unpredictability of child welfare and how to respond to the many unexpected challenges that may arise. Finally, the relationship between competency and burnout or turnover is not clearly understood. Future research should focus on the application of skills, knowledge, and values attained through the Title IV-E program in IV-E graduates’ practice as full-time child welfare employees, and on the outcomes for the children and families they serve. References Adoption Assistance and Child Welfare Act of 1980. P.L. 96-272. §§ 422(1980). Bagdasaryan, S. (2012). Social work education and Title IV-E program participation as predictors of entry-level knowledge among public child welfare workers. Children and Youth Services Review, 34(9), 1590-1597. doi: https://doi.org/10.1016/j.childyouth.2012.04.013 Barth, R. P., Lloyd, E. C., Christ, S. L., Chapman, M. V., & Dickinson, N. S. (2008). Child welfare worker characteristics and job satisfaction: A national study. Social Work, 53(3), 199-209. doi: https://doi.org/10.1093/sw/53.3.199 Cheung, C., & Tang, K. (2010). Socialization factors conducive to social work students’ gain in competence: Experience in the Hong Kong SAR, China. Journal of Social Work, 10(1), 42-58. doi: https://doi.org/10.1177/1468017309349990 Council on Social Work Education [CSWE]. (2015). Educational policy and accreditation standards for baccalaureate and master’s social work programs. CSWE on Accreditation. Retrieved from http://www.cswe.org/file.aspx?id=81660 Dillman, D. A., Smyth, J. D., & Christian, L. M. (2008). Internet, mail, and mixed-mode surveys: The tailored design method (3rd ed.). New York, NY: Wiley. Fox, S. R., Miller, V. P., & Barbee, A. P. (2003). Finding and keeping child welfare workers: Effective use of training and professional development. Journal of Human Behavior in the Social Environment, 1(2), 67-81. doi: https://doi.org/10.1300/J137v07n01_06 Franke, T., Bagdasaryan, S., & Furman, W. (2009). A multivariate analysis of training, education, and readiness for public child welfare practice. Children and Youth Services Review, 31(12), 1330-1336. doi: https://doi.org/10.1016/j.childyouth.2009.06.004 Gansle, K. A., & Ellett, A. J. (2002). Child welfare knowledge transmission, practitioner retention, and university community impact: A study of Title IV-E child welfare training. Journal of Health & Social Policy, 15(3/4), 69-88. doi: https://doi.org/10.1300/J045v15n03_06 Government Accountability Office. (2003). Child welfare: HHS could play a greater role in helping child welfare agencies recruit and retain staff [GAO-03-357]. Washington, DC: Author. Retrieved from http://www.gao.gov/products/GAO-03-357 https://doi.org/10.1016/j.childyouth.2012.04.013 https://doi.org/10.1093/sw/53.3.199 https://doi.org/10.1177/1468017309349990 http://www.cswe.org/file.aspx?id=81660 https://doi.org/10.1300/J137v07n01_06 https://doi.org/10.1016/j.childyouth.2009.06.004 https://doi.org/10.1300/J045v15n03_06 http://www.gao.gov/products/GAO-03-357 ADVANCES IN SOCIAL WORK, Fall 2017, 18(2) 489 Hartinger-Saunders, R. M., & Lyons, P. (2013). Social work education and public child welfare: A review of the peer-reviewed literature on Title IV-E funded programs. Journal of Public Child Welfare, 7(3), 275-297. doi: https://doi.org/10.1080/15548732.2013.798246 Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14-26. doi: https://doi.org/10.3102/0013189X033007014 Jones, L. P., & Okamura, A. (2000). Reprofessionalizing child welfare services: An evaluation of a Title IV-E training program. Research on Social Work Practice, 10(5), 607-621. Krueger R A. (1988). Focus groups: A practical guide for applied research. New Dehli, India: Sage. Krueger, R. A., & Casey, M. A. (2009). Focus groups: A practical guide for applied research. London, United Kingdom: Sage. QSR International Pty Ltd. (2010). NVivo qualitative data analysis software [Version 9, Windows]. Victoria, Australia. Padgett, D. K. (Ed.). (2004). The qualitative research experience. Pacific Grove, CA: Wadsworth/Thomson Learning. Pelfrey, W. V., Sr., & Pelfrey, W. V., Jr. (2009). Curriculum evaluation and revision in a nascent field: The utility of the retrospective pretest-posttest model in a homeland security program of study. Evaluation Review, 1, 54-82. doi: https://doi.org/10.1177/0193841X08327578 Pratt, C. C., McGuigan, W. M., & Katzev, A. R. (2000). Measuring program outcomes: Using retrospective pretest methodology. American Journal of Evaluation, 21, 341- 349. doi: https://doi.org/10.1177/109821400002100305 Yankeelov, P. A., Barbee, A. P., Sullivan, D., & Antle, B. F. (2009). Individual and organizational factors in job retention in Kentucky's child welfare agency. Children and Youth Services Review, 31, 547-554. doi: https://doi.org/10.1016/j.childyouth.2008.10.014 Zlotnik, J. L. (1997). Social work education programs, state child welfare agencies comment on Title IV-E regulations. Social Work Education Reporter, 4(6), 8-11. Zlotnik, J., DePanfilis, D., Daining, C., & McDermott Lane, M. (2005). Professional education for child welfare practice: Improving retention in public child welfare agencies. Institute for the Advancement of Social Work Research, IASWR Research Brief 2. Retrieved from https://archive.hshsl.umaryland.edu/handle/10713/289 Author note: Address correspondence to: Elizabeth J. Greeno, PhD University of Maryland School of Social Work, 525 W. Redwood Street, Baltimore, MD 21201. egreeno@ssw.umaryland.edu. https://doi.org/10.1080/15548732.2013.798246 https://doi.org/10.3102/0013189X033007014 https://doi.org/10.1177/0193841X08327578 https://doi.org/10.1177/109821400002100305 https://doi.org/10.1016/j.childyouth.2008.10.014 https://archive.hshsl.umaryland.edu/handle/10713/289 Elizabeth Jane Greeno Lisa Fedina Berenice Rushovich Caroline Burry Debra Linsenmeyer Christopher Wirt Child Welfare Knowledge and Skill Acquisition Previous Child Welfare Experience University of Maryland, Baltimore Title IV-E Program Purpose of Study Methods Sampling and Recruitment Data Collection Procedures Results Demographics Retrospective Pretest Findings Demographic Differences Prior Child Welfare Experience Preparation to Practice Post-Graduation Work and Life Balance Discussion Limitations Conclusion and Recommendations References