Running head: Child Welfare Workers’ Connectivity Child Welfare Workers’ Connectivity to Resources and Youth’s Receipt of Services Alicia C. Bunger Arlene R. Stiffman Kirk A. Foster Peichang Shi Abstract: Youth involved in the child welfare system are at high risk for mental illness, substance abuse, and other behavioral health issues, which child welfare workers are expected to address through referrals. Child welfare workers (N=27) who participated in Project IMPROVE (Intervention for Multisector Provider Enhancement) reported on services they provided to youth (N=307) in their caseloads. Using survey and administrative data, this paper examines workers’ service actions on behalf of youth. Results were consistent with the Gateway Provider Model and showed that youth received help from a greater variety of service sectors when their workers were able to identify behavioral health problems, and were familiar with and connected to other providers in the community. Improving service delivery to youth in child welfare may be accomplished by training workers in the signs and symptoms of behavioral health problems and familiarizing them with providers in the community. Keywords: Child welfare workers; behavioral health; connectivity INTRODUCTION Child welfare workers are responsible for advancing the safety, permanency, and well-being of youth involved in the child welfare system. Consistent with federal child welfare goals, workers must also help youth obtain treatment and support services (U.S. Department of Health and Human Services, 2003) requiring them to reach across the traditional boundaries of the child welfare system to other human service systems. Because the estimated prevalence of behavioral health conditions is greater for youth in the child welfare system than in the general public assistance sector, workers’ connections with behavioral health services are particularly important (Burns, et al., 2004; Harman, Childs, & Kelleher, 2000; Leslie, Hurlburt, Landsverk, Barth & Slymen, 2004). Workers in public child welfare agencies are well-positioned to identify behavioral health service needs among youth and connect them to appropriate care in the community, thus serving as Gateway Providers to mental health, substance abuse, and other service sectors (Stiffman, Pescosolido & Cabassa, 2004). This paper examines the predictors of child welfare workers’ service actions for youth with behavioral health _________________ Alicia C. Bunger is a doctoral student at the Brown School of Social Work in St. Louis, MO. Arlene R. Stiffman, Ph.D., is the Barbara A. Bailey Professor of Social Work, Emeritus and Director of the NIDA Comorbidity and Addictions Center; Kirk A. Foster is a doctoral student and NIDA Comorbidity and Addictions Center Administrator; and Peichang Shi is a statistical data specialist, all at the Brown School of Social Work. This research was supported by NIMH grants R34 MH072871, P30 MH068759, T32 MH19960 and K02 MH001797, and NIDA grant R24 DA013572. Special thanks to Jennifer Bellamy for comments on earlier drafts of this paper. Copyright © 2009 Advances in Social Work Vol. 10 No. 1 (Spring 2009), 19-38 Bunger, Stiffman, Foster, Shi/CHILD WELFARE WORKERS’ CONNECTIVITY 20 problems from the context of the Gateway Provider Model (Stiffman, Pescosolido et al., 2004). Behavioral Health Problems and Services Use Youth who experience traumatic and adverse experiences like child abuse and neglect are at high risk for psychiatric disorders, social problems, and general functional impairment (Cerezo & Frias, 1994; Famularo, Kinscherff, & Fenton, 1992; Kaufman, 1991; Kazdin, Moser, Colbus, & Bell, 1985). Youth involved in the child welfare system have high rates of behavioral health problems with estimated prevalences ranging from 50-70% (Burns, et al. 2004; Garland,et al., 2001; Trupin, Tarico, Low, Jemelka, & McClellan, 1993) compared to 20% in the general youth population (Costello et al., 1996; Shaffer, et al., 1996). Behavior problems and conduct disorders are particularly prevalent but depression, anxiety, post-traumatic stress, and substance use disorders are also common (Garland et al., 2001; Halfon, Berkowitz & Klee, 1992; Stiffman, Chen, Elze, Doré & Cheng, 1997). Youth in the child welfare system use behavioral health services at high rates. State Medicaid programs and child welfare agencies finance a range of behavioral health services that include inpatient, outpatient, and residential care. Studies of mental health service users show that youth in child welfare, particularly those in out-of-home placements, use these behavioral health services at a much higher rate than youth in the community (Farmer et al., 2001; Halfon et al., 1992; Harman et al., 2000). Youth’s need is a strong predictor of their use of behavioral health services. Although research supports the relationship between need and receipt of behavioral health services, evidence suggests that other factors such as abuse type, placement type, age, and race/ethnicity also influence youth’s receipt of services (Garland, Landsverk, Hough & Ellis-MacLeod, 1996; Hurlburt et al., 2004; Leslie et al., 2000; Walrath, Ybarra, Sheehan, Holden & Burns, 2006). In addition, child welfare workers’ daily decisions and actions on behalf of the youth they serve also have the potential to impact the services youth receive. Role of Child Welfare Workers in Youth Access to Care Child welfare workers impact the services, resources, and placements youth receive when involved with the system (Ryan, Garnier, Zyphur, & Zhai, 2005). Specifically, youth’s entry into the behavioral health service sector has been linked to their contact with child welfare services (Leslie, Hurlburt et al., 2005; Stiffman, Chen, Elze, Doré & Cheng, 1997). However, poor assessment of treatment needs and limited access to behavioral health services are barriers in youth’s pathways between the child welfare and behavioral health systems, leaving many with unmet service needs (Leslie, Gordon, et al., 2005; Stiffman, Pescosolido et al., 2004). Research on systems of care suggests that provider-level actions are important determinants of care regardless of administrative reforms designed to bridge systems (Bickman, 1996; Bickman, Lambert, Andrade & Penaloza, 2000; Bickman, Noser & Summerfelt, 1999). The Gateway Provider Model posits that non-mental health providers play a key role in connecting youth to services (Stiffman, Pescosolido et al., 2004). To ADVANCES IN SOCIAL WORK, Spring 2009, 10(1) 21 ensure that youth receive treatment, gateway providers must be able to recognize behavioral health service needs and be familiar with and connected to the larger service network. Therefore, connectivity, or the degree to which a provider interacts with youth- serving agencies can influence services youth receive (Stiffman, Hadley-Ives et al., 2000; Stiffman et al., 2001). This paper examines the relationship between worker knowledge and skills and behavioral health service provision to youth. We believe it is the first to explore the applicability of this framework for child welfare workers, while verifying it with state billing data. Based on evidence with providers from multiple disciplines (Stiffman, Hadley-Ives et al., 2000; Stiffman, Pescosolido et al., 2004), we posited that child welfare workers’ connectivity and recognition of behavioral health service needs would be positively associated with service provision (Figure 1). Figure 1. Selected Elements of the Gateway Provider Model of Service Provision Gateway Provider Skills and Knowledge •Assessment of youth mental health and environment • Resource Knowledge (connectivity) • Familiarity • Referrals Provider Type and Training • Experience • Education • Training Service Provision Decision to Provide Services • Service Actions METHOD Design Project IMPROVE (Intervention for Multisector Provider Enhancement) was an intervention study funded through the National Institute of Mental Health (NIMH). Through the intervention, the study examined child welfare worker knowledge of their clients’ needs, their referral and treatment actions on behalf of those clients, and also obtained billing records concerning diagnoses and services provided to these youth (Stiffman, Foster, Hamburg, & Doré, 2003). The analyses for this paper utilized state billing records and baseline quantitative survey data from the larger IMPROVE intervention study. Bunger, Stiffman, Foster, Shi/CHILD WELFARE WORKERS’ CONNECTIVITY 22 Sample Child welfare workers (N=27) in the St. Louis City and County Children’s Division, Missouri Department of Social Services self-selected into the study, providing they had an active case load of at least 15 youth between five and 17 years of age and a job description that included mental health and substance abuse assessment and referral as regular job duties. As part of the intervention, workers were invited to the university for training and data collection. While onsite, workers completed self-administered surveys where they provided data on 307 youth whom they identified only though their public record locator code (the Departmental Client Number or DCN). Both the university’s Institutional Review Board and the Missouri Department of Social Services Privacy Review Board approved the protocol and each worker provided written consent. The review boards waived third party consent for workers’ reports on the youth because youth names were never available to the study, and all other youth data were de-identified prior to the study’s access. Workers in the sample were predominantly female (85%) and white (66%). All workers possessed a bachelors-level degree with 15% having a masters-level degree. The average worker was employed by the Children’s Division for three years (SD=2.14), with a range from less than one year to seven years. Measures Measures consisted of worker reports and state administrative data. Worker Reports Workers reported on (1) their background and knowledge of community referral resources (connectivity); (2) youth’s mental health and environmental problems; and (3) services obtained for youth, via self-administered surveys comprised of measures used successfully in several prior studies (Horwitz et al., 2001; Stiffman et al., 2006; Stiffman, Hadley-Ives et al., 2000; Stiffman, Horwitz et al., 2000; Stiffman et al., 2001), Worker Background Education and Training. Workers reported the highest level of education they obtained, the total hours of in-service training or continuing education in the past year, and the types of topics covered in the training (e.g. illicit drug use, assessment, neglect). Connectivity. Provider connectivity is the degree to which an individual worker is connected to other youth-serving agencies (Stiffman et al., 2006; Stiffman, Hadley-Ives et al., 2000; Stiffman et al., 2001). Workers reported their relationships with twenty-eight service categories grouped into four major service domains (Appendix A): six for the health and education domain (e.g., job training, tutoring, public health centers); seven for the inpatient behavioral health care domain (e.g., psychiatric or substance abuse treatment facilities); eight for the outpatient treatment domain (e.g., community-based mental health or substance abuse services); and seven for the ‘Other’ service domain (e.g., religious providers, basic needs). ADVANCES IN SOCIAL WORK, Spring 2009, 10(1) 23 Workers reported on two types of relationships: familiarity and referrals (for any youth in their caseload, not just those reported on in this study) in the past six months. The data provided three measures: 1) the number of service categories with which workers were familiar (familiarity); 2) the sum of referrals made to service categories (referrals); and 3) an aggregate score of the sum total of both familiarity and referrals (total connectivity). Worker Reports on Youth Workers reported on their perceptions of youth’s behavioral health and environmental problems. Behavioral Health Problems. We measured workers’ perceptions of behavioral health problems using eight questions concerning depression, post traumatic stress, anxiety, alcohol misuse, illicit drug use, behavior problems/conduct disorder, and other (Stiffman et al., 2006; Stiffman, Hadley-Ives et al., 2000; Stiffman et al., 2001). Youth could have between zero and eight behavioral health problems. Behavioral health problems were assigned if the worker perceived the problem to be moderate, serious, or critical. Environmental Problems. Workers reported on the presence or absence of 14 types of environmental or social problems including school and learning problems, physical health problems, basic need issues, problem peers, violence in the family, family problems, living in a violent neighborhood, exposure to violence in school, gangs, family instability, not living at home, lack of family support, legal problems, and family financial problems (Stiffman et al., 2006; Stiffman, Hadley-Ives et al., 2000; Stiffman et al., 2001). Service Provision We collected information about workers’ service actions on behalf of youth, and the services youth obtained. Service Actions. The number of services that workers provided directly to youth, and the number of services referred were measured (Stiffman, Hadley-Ives et al., 2000, Stiffman et al., 2001). Workers reported whether they acted in any of four ways (took no action; recommended or referred youth to service; personally provided but did not refer; or personally provided and referred) for 11 types of services: parenting/caregiving, counseling/therapy, teaching/alternative/education-related services, substance abuse treatment, self-help group or peer counseling, crisis intervention, inpatient/residential mental health care, medication to control symptoms, psychiatric evaluation, family counseling, and service coordination and referral. Services Obtained. Workers completed the brief module of the Services Assessment for Children and Adolescents (SACA) which collects information about inpatient, outpatient, school and other services used in the last six months, whether or not they were provided by the child welfare workers (Horwitz et al., 2001; Stiffman, Horwitz et al., 2000). Bunger, Stiffman, Foster, Shi/CHILD WELFARE WORKERS’ CONNECTIVITY 24 Administrative Data Billing records from the Missouri Division of Mental Health and Medicaid were extracted for each youth reported on by the workers. These records were included because they reflect outcomes of child welfare service actions - child welfare workers refer youth to outside service providers who, in turn, bill the Division of Mental Health or Medicaid for services rendered. We used DCN numbers to merge the records from both sources. The data included information about services billed for the youth between July 1, 2005 and March 8, 2008. Each billing record contained a diagnostic code indicating the type of problem the youth was treated for, and a procedure code indicating the type of service provided. Diagnostic Codes The data contained ICD-9 behavioral health diagnostic codes and their descriptions. We grouped the diagnostic codes into the same eight types of problems that workers reported in the survey (depression, post traumatic stress, anxiety, alcohol misuse, illicit drug use, behavior problems/conduct disorder, and other). Procedure Codes Procedure codes followed the Common Procedural Terminology (CPT) and Healthcare Common Procedure Coding System (HCPCS) conventions. We obtained descriptions of the CPT and HCPCS codes and grouped them into four main types of services: overnight, outpatient, school-based, and other. Procedure categories (like the diagnostic categories) were grouped to parallel the types of services reported through the SACA. Analyses Two forms of analysis were performed: 1) accuracy calculations (the percentage of all cases in which both data sources agreed that the youth had a problem or service or did not have a problem or service) to examine the relationship between provider reports on youth and the billing data; and 2) generalized estimating equations (GEE) with a negative binomial distribution to examine multivariate predictors of service provision. Our data did not satisfy the independence and normality assumptions of general linear regression. First, our data was nested (307 youth clustered by 27 workers) and the reports about youth with the same worker were related observations, and not independent. Second, the dependent variable (services obtained as reported on the SACA) was not normally distributed. As is common when studying services or other counts of rare events, the data took the shape of a negative binomial distribution (Gardner, Mulvey, & Shaw, 1995), where the majority of observations were concentrated at the low end (none or only a few types of services), and only a few observations fell along the upper range (six or more types of services). Therefore, GEE is an appropriate method to analyze the non-normal (negative binomial) distribution of the dependent variable and to account for correlations within worker subgroups (Ballinger, 2004). The model fit was determined by a Pearson chi square test where a lack of significant difference between the observed and expected values indicated a good fit. Analyses were run using SAS 9.1 and Stata SE 9. ADVANCES IN SOCIAL WORK, Spring 2009, 10(1) 25 RESULTS We examined information about youth’s demographic characteristics, the relationships between worker reports and administrative data on youth’s behavioral health problems, environmental problems, and services received, and workers’ connectivity to community resources. Then we explored the Gateway Provider Model’s assumptions about predictors of service provision. Youth Demographic Characteristics (based on worker reports) The youth were nearly evenly split between males (49%) and females (51%). The age of the youth ranged from five to 20 (median of 13 years). Over two-thirds of youth in the sample were African-American (70.9%), about one-quarter were Caucasian (28.8%), 1.5% Hispanic, and 1.2% were “other.” Relationship between Worker Reports and Administrative Data Youth’s behavioral health and environmental problems, and service provision reported by workers and documented in the administrative data were examined separately. Then the two data sources were compared to determine the correspondence between both types of reports. Behavioral Health Problems We examined youth’s behavioral health problems by analyzing worker reports and the diagnostic codes that accompanied the billing records in the administrative data. Findings from each data source were then compared (Table 1). Worker Reports of Behavioral Health Problems. Workers reported that nearly two- thirds (64%) of youth in each of their caseloads had a moderate to severe behavioral health problem. The two most commonly reported were depression (44%) and behavior problems/conduct disorder (43%), followed by anxiety (33%), post traumatic stress (29%), illicit drug use (5%), suicidality (4%), and alcohol misuse (2%). About 10% of youth had “other” types of behavioral health problems such as developmental delays and personality disorders. Billing Records on Behavioral Health Problems. Like the worker reports, billing records showed that nearly three quarters (73%) of youth were diagnosed with some type of behavioral health problem. Similarly, the most common diagnoses were behavior problems/conduct disorder (57%) and depression (41%), followed by anxiety (18%), post-traumatic stress (11%), illicit drug use (10%), suicidality (1%), and alcohol misuse (1%). Unlike the worker reports, a much higher percentage of youth had “other” types of behavioral health problems. Approximately 50% of youth were diagnosed with psychotic and personality disorders, developmental delays, or other disorders not otherwise specified. Relationship between Worker Reports and Billing Records for Behavioral Health Problems. Behavioral health diagnoses in the billing records and worker reports on whether or not youth had a behavioral health problem matched 69% of the time. The highest concordance between worker reports and billing records were for alcohol misuse Bunger, Stiffman, Foster, Shi/CHILD WELFARE WORKERS’ CONNECTIVITY 26 (97%), suicidality (96%), and illicit drug use (89%), followed by post traumatic stress (72%), behavior problems/conduct disorder (65%), anxiety (63%), depression (62%) and other (52%). Table 1. Youth Behavioral Health Problems Reported by Workers and Billing Data (N=307) Behavioral Health Problem Worker Report % Billing Data % Concordance % Depression 44 41 62 Behavior Problems/Conduct Disorder 43 57 65 Anxiety 33 18 63 Post-traumatic Stress 29 11 72 Illicit Drug Use 5 10 89 Suicidality 4 1 96 Alcohol Misuse 2 1 97 Other 10 50 52 Total 64 73 69 It is unlikely that two data sources will ever be in total concordance, therefore some discordance between worker reports and billing data is to be expected. These data may not agree because of differences in perception of the problem between workers and providers, as well as differing reporting, and billing policies. Environmental Problems First we explored worker reports of environmental problems and second we examined the diagnostic codes that accompanied the billing records in the administrative data. Findings from each data source were then compared (Table 2). Worker Reports of Environmental Problems. Almost all (98%) of the youth had environmental problems, which is not surprising for public child welfare clients (for example, see Berger, 2004 or Hardin & Koblinski, 1999). Workers most frequently noted the following environmental problems among youth in their caseload: family instability (78%), family problems (78%), not living at home (67%), and school and learning (66%). These were followed by lack of family support (53%), basic need issues (52%), family financial problems (41%), problem peers (38%), violence in the family (30%), physical health problems (14%), violent neighborhoods (13%), violent schools (12%), legal problems (9%), and gangs (7%). Billing Records for Environmental Problems. Billing records showed much lower rates of environmental problems than worker reports. These results are to be expected since billing records reflect service needs reimbursed by the state which are typically psychiatric or health-related rather than environmental. Almost 21% of youth were diagnosed with some type of environmental problem. About 10% received a diagnostic ADVANCES IN SOCIAL WORK, Spring 2009, 10(1) 27 code for violence in the family, 7% with family problems, 4% with basic needs, 2% with school and learning problems and 1% with exposure to violence in school. Relationship between Worker Reports and Billing Records on Environmental Problems. The diagnostic codes in the billing records matched worker reports of environmental problems (those that are billable) 65% of the time. Exposure to violence in schools had the highest match rate (83%) followed by violence in the family (65%), basic needs (49%), school and learning problems (35%), and family problems (27%). Table 2. Youth Environmental Problems Reported by Workers and Billing Data (N=307) Environmental Problems with billing codes Worker Report % Billing Data % Concordance % Family Problems 78 7 27 School and Learning 66 2 35 Basic Needs 52 4 49 Violence in the Family 30 10 65 Violent Schools 12 1 83 Total (for problems with billing codes) 94 21 65 Environmental Problems without billing codes Family Instability 78 -- -- Not Living at Home 67 -- -- Lack of Family Support 53 -- -- Family Financial Problems 41 -- -- Problem Peers 38 -- -- Physical Health Problems 14 -- -- Violent Neighborhoods 13 -- -- Legal Problems 9 -- -- Gangs 7 -- -- Total 98 -- -- Service Provision We explored three types of service reports: (1) worker reports of personal service actions on behalf of youth, (2) worker reports of any youth services according to the SACA, and (3) procedure codes for services in the billing records. The agreement between the SACA service data and billing records was also calculated (Table 3). Worker Reports – Service Actions. Workers reported their personal service actions on behalf of the youth. Workers provided and/or referred 84% of youth to services. For the most part, when workers identified youth’s need for service, they responded by either Bunger, Stiffman, Foster, Shi/CHILD WELFARE WORKERS’ CONNECTIVITY 28 providing the service or through referral. On average, workers did not take action for 12% of youth’s identified service needs. Most commonly, workers provided or referred youth for counseling/therapy services (67%) followed by service coordination and referral (57%), and education-related services (50%). Other services included family counseling (33%), medications for symptom management (26%), psychiatric evaluation (25%), crisis intervention (21%), self help or peer groups (20%), parenting/care-giving (17%), inpatient mental health care (14%), and substance abuse treatment (7%) (Figure 2). Figure 2. Service Actions on Behalf of Youth 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Pare nti ng /Care giv ing Cou nse lin g o r T her apy Edu cat ion Sub sta nc e A bu se Tx Self -H elp Cris is I nte rve nti on Inp ati ent M H Care Med ica tio n Psyc h E val Fam ily Cou nse lin g Svc Coo rdi na tio n Services Pe rc en t o f y ou th Needed, Action Taken Needed, No Action Not Needed Worker Reports – SACA. Workers reported that youth and their families received services from an average of four different service sectors within the previous six months. The most commonly received services by youth include incidentals/clothing/transportation (68%), school-based help for problems with behavioral health issues (47%) and recreational/community activities (46%). According to workers, about 38% of youth received help from outpatient facilities such as a mental health clinic, and 23% received overnight help from a facility such as a hospital, group home, or foster home for problems with drugs, alcohol, or emotional problems. Billing Records of Services. Similar to worker reports, administrative data showed that 88% of youth had a procedure code in the billing records for services. Unlike the SACA data, only 12% had records for incidentals/clothing/transportation. But once again, as with environmental issues, these types of services are seldom billed to Medicaid or the Division of Mental Health. Most youth (88%) had records for outpatient services, 15% had charges for overnight services. ADVANCES IN SOCIAL WORK, Spring 2009, 10(1) 29 Relationship between Worker Reports and Billing Records of Services. The presence of worker reports and procedure codes in the billing records matched 80% overall. The highest match rate was for overnight care (76%), followed by outpatient services (45%), and incidentals/clothing/transportation (38%). Table 3. Services Provided Reported by Workers and Billing Data Service Sector Worker Report % Billing Data % Concordance % Incidentals/Clothing Transportation 68 12 38 School-based Help 47 -- -- Recreational/Community Activities 46 -- -- Outpatient 38 88 45 Inpatient/Overnight 23 15 76 Overall Match (services v. no services) 86 88 80 Connectivity Scores Workers’ connectivity to referral resources in the community is a key construct examined in this study. Total connectivity scores (familiarity and number of referrals) ranged from 37 to 272. The average was 83 with a median of 64, indicating that the distribution was positively skewed by several extreme outliers. There was no statistically significant relationship between worker education, number of training hours, or type of training and connectivity. Each component of the score is described below and summarized in Table 4. Familiarity Workers are familiar with at least some organizations in most service domains. All of the workers were familiar with at least one resource in each of the four domains and almost half (47%) were familiar with all 28 service categories. On average, workers were familiar with 26 of the 28, or about 93% of the service categories. Referrals Workers made referrals across all four major domains of services in the six months prior to the intervention. The number of total referrals ranged from 15 to 245. The average number of referrals was 57 and the median was 37, indicating large variations in referral patterns. Twenty six percent of referrals were made to outpatient services, 25% to inpatient services, and 19% to health and education services, with the greatest (29%) to “Other” service domains. Bunger, Stiffman, Foster, Shi/CHILD WELFARE WORKERS’ CONNECTIVITY 30 Table 4. Familiarity, Referrals and Connectivity by Domain Baseline Domain Familiar % Referrals X̄/(Median) Connectivity X̄/(Median) Health & Education 92 10.7/(8.0) -- Inpatient 93 14.4/(8.0) -- Outpatient 93 14.9/(11.0) -- Other 94 16.6/(14.0) -- Total 93.2 56.7/(37.0) 82.7/(64.0) Model Exploration Above, we examined the separate constructs of the Gateway Provider Model which posits that providers’ connectivity to the larger service network and ability to identify behavioral health service needs influences youth’s receipt of behavioral health services. Here we examine the multivariate relationships between those variables and service provision. Multivariate Predictors of Service Provision. The Gateway Provider Model predicts that provider assessment of behavioral health problems and connectivity determines provision of care. The multivariate model is based only on worker reports of behavioral health problems, connectivity and services; billing records are not included in this analysis because they reflect the billable actions of multiple service providers, not the individual child welfare actions that are the focus of this study. As hypothesized, when all key variables were entered into the equation, workers’ assessment of mental health and workers’ connectivity together contributed to the prediction of SACA reports of the number of sectors of care that served youth. However, assessment of environmental problems was not significantly related to service receipt. In our final model, where we retained only the significant predictors, workers’ assessment of mental health problems (b=.2923, SE=.0664, p<.0001) and workers’ connectivity (b=.0026, SE=.0009, p=.0048) predicted 37% of the variance (marginal R2) in service receipt. A goodness of fit test demonstrated an adequate model fit, χ2(253)=260.12, p=.37. We ran a similar analysis to examine whether workers’ assessment of behavioral problems and connectivity predict the number of sectors of care that served youth in the billing records. None of the independent variables significantly predicted the number of sectors of care reported in the billing records. ADVANCES IN SOCIAL WORK, Spring 2009, 10(1) 31 DISCUSSION This paper addressed the influence of child welfare workers’ ability to recognize mental health problems and the effect their knowledge of community referral resources had on behavioral health service provision to youth. Youth involved in the Missouri Children’s Division received services from multiple sectors of care, reinforcing the importance of child welfare workers’ role as gateways to care for youth. As expected, child welfare workers’ connectivity and their identification of mental health problems were associated with youth’s receipt of services and explained 37% of the variance in youth’s service receipt. These two worker-level characteristics had more explanatory power than Andersen’s model which is usually found to explain about 20% of the variance in service use (Mechanic, 1979; Phillips, Morrison, Andersen & Aday, 1998). Youth receive help from a greater variety of service sectors when their gateway providers are able to identify mental health problems and are familiar with and connected to community resources. For example, one way of thinking about these relationships is that if a youth with two types of mental health problems is working with a child welfare worker with a connectivity score of 131, she or he will likely receive care from an average of six different service sectors. However, if the same youth is working with a child welfare worker with a connectivity score of 74, she or he will likely receive care from about five different service sectors (assuming that both child welfare workers similarly identify youth’s mental health problems). These are important findings because youth in the child welfare system have multiple behavioral health and environmental problems requiring services from multiple service sectors. While more service referrals may not benefit an individual client, the more service sectors involved in their care increases the likelihood that a greater number of service needs are addressed, which is the ultimate goal of the call for inter-agency coordination and reduced fragmentation (Stroul & Friedman, 1986). Identification of Youth’s Problems Our data show that workers’ ability to identify mental health problems among youth in their caseload is critical for service delivery. Workers reported that most of the youth had moderate to severe mental health problems. Behavior problems and mood disorders were most common among youth while substance misuse and suicidality were less frequent. Workers identified environmental problems for nearly all of the youth. The majority of these problems were related to families and school. Although workers’ identification of environmental problems was not significantly related to service provision in multivariate analyses, the descriptive information about the types of mental health and environmental problems illustrates the multitude of complex problems that child welfare workers address on behalf of youth. Worker Response to Youth’s Problems Workers responded to mental health problems with a variety of services. The service action data suggests that workers address youth’s mental health problems by recommending counseling and therapy, service coordination, and education-related Bunger, Stiffman, Foster, Shi/CHILD WELFARE WORKERS’ CONNECTIVITY 32 services. Their connections with community agencies that provide these services reveal additional insight about referral practices that guide youth in the child welfare system to the behavioral health service system. Connectivity Data show that workers are familiar with nearly all of the service domains and make frequent linkages with other organizations. Workers often connected with general social service agencies in the community to meet youth’s basic needs. However, workers also refer frequently to outpatient and inpatient services specifically designed to address behavioral health needs. The wide range of reported referrals to and familiarity with youth-serving agencies suggest that workers vary in the way they are connected with the full inter-organizational network of agencies in the region. Neither education, experience, nor training accounted for the variations in connectivity, suggesting that other factors determine workers’ knowledge and connections to referral resources. Understanding these variations in connectivity is important because workers’ connections to resources facilitated youth’s receipt of services. Our results provide an opportunity to advance a practice-oriented research agenda focusing on individual worker characteristics, practices, and skills that help bridge multiple service delivery systems. Limitations The primary findings of this study were based on workers’ reports on youth so we were concerned that recall biases may influence the data. To evaluate the potential impact of such a bias, we compared worker reports with state billing records. If there was little concordance between billing records and worker reports, our findings in support of the model might have been spurious. However, the billing records support the general reliability of worker assessments and reports of service provision. Both worker reports and billing data were consistent in reports of mental health problems, particularly conduct disorder/behavior problems and depression. The high correspondence between worker reports of service receipt and billing records bolstered our confidence in the reliability of the SACA data. The discrepancy between worker reports and the billing records on youth’s environmental problems may be attributed to an underestimation of environmental problems in the billing records in our study because CPT or HCPCS codes do not exist for all of the types of environmental problems measured in the workers’ survey instrument, and the state may not reimburse providers for services related to all of types of environmental problems. The discrepancy does raise questions regarding youth’s access to social services, and how organizations finance the provision of services that address environmental problems among youth in the child welfare system. Our worker reported data indicate that youth obtained services (e.g. transportation and incidentals) that could address some environmental problems (e.g. basic needs and family financial problems). However, we cannot draw any conclusions about whether and how youth- serving agencies in the community address other environmental problems like family violence or instability. This gap suggests that child welfare workers need to be familiar ADVANCES IN SOCIAL WORK, Spring 2009, 10(1) 33 with youth-serving agencies that are able to provide non-billable services in addition to those with formal contracts or the ability to bill public systems for services. Neither worker self-reports nor billing records are gold standard measures for behavioral health or environmental problem diagnoses and there is no perfect measure for capturing service receipt. However, we can present our data and results here with greater confidence because the billing data and worker reports of youth’s problems and services are fairly consistent. The study focused on practices within public child welfare offices in St Louis, MO and findings may not be generalizable across other locales. Other factors not included in this study may influence assessment, referral practices and service provision for youth. Studies conducted with the St. Louis Children’s Division (Fedoravicius, McMillen, Rowe, Kagotho & Ware, 2008; Foster & Stiffman, 2009; McMillen, Fedoravicius, Rowe, Zima & Ware, 2007) show that mental health services for youth in the child welfare system are frequently court ordered. The pressure to comply with court demands for documentation and time constraints influence workers’ assessment and referral practices (Fedoravicius et al., 2008; Foster & Stiffman, 2009; Smith & Donovan, 2003). The small sample size did not allow us to run a structural equation model to test the Gateway Provider Model, the preferred method to examine such relationships. Social desirability and recall biases may have influenced the accuracy of the connectivity data reported. Workers may have felt pressure to appear well-connected or may have had difficulty remembering the number of referrals they made. Finally, the continuity of services was not addressed in the data. We cannot determine whether youth received continuous services from a single provider, or sporadically by many providers. Despite these limitations, the study findings provide support and direction for the growing body of literature on the role of child welfare workers’ knowledge and practices on youth entry into the behavioral health service system. Future Studies Given the variation in workers’ connectivity, the next step in this line of research is to examine how workers develop their knowledge of referral resources. If training, experience and education were not associated with connectivity, other factors that were not included in these analyses must be important. Focus groups revealed that interactions at the office shaped workers’ familiarity with other providers and their referral patterns (Foster & Stiffman, 2009). Constructive organizational culture and positive perceptions of organizational climate influence work-place interactions and may have a role in the transfer of practice knowledge (Glisson & Green, 2006; Uzzi & Lancaster, 2003). The Gateway Provider Model also posits that organizational culture and climate are related to connectivity. Exploring and testing these relationships could inform training in child welfare agencies and have implications for cross-sector service delivery to youth served by the system. Bunger, Stiffman, Foster, Shi/CHILD WELFARE WORKERS’ CONNECTIVITY 34 CONCLUSION This study highlights how child welfare workers act as gateway providers and influence youth’s receipt of behavioral health services. The identification of behavioral health problems and connections to service providers who can meet youth’s needs are critical skills and knowledge for child welfare workers. These skills might also be important for social workers and other human service professionals who are responsible for working across system boundaries. Child welfare workers’ ability to identify behavioral health problems and their knowledge of service providers in the community make a difference. Given the high prevalence of behavioral health problems among youth involved in the child welfare system, it is important to identify and provide clear pathways to needed treatment for youth. By improving providers’ connections with agencies in their community and their ability to identify behavioral health problems, child welfare systems may enhance youth’s service access and receipt and reduce unmet needs for behavioral health services. References Ballinger, G. A. (2004). Using generalized estimating equations for longitudinal data analysis. Organizational Research Methods, 7(2), 127-150. Bickman, L. (1996). A continuum of care: More is not always better. American Psychologist, 51(7), 689-701. Bickman, L., Lambert, W., Andrade, A.R., & Penaloza, R.V. (2000). The Fort Bragg continuum of care for children and adolescents: Mental health outcomes over 5 years. Journal of Consulting and Clinical Psychology, 68(4), 710-716. Bickman, L., Noser, K., & Summerfelt, W. T. (1999). Long-term effects of a system of care on children and adolescents. Journal of Behavioral Health Services and Research, 26(2), 185-202. Burns, B. J., Phillips, S., Wagner, H. R., Barth, R. P., Kolko, D. J., Campbell, Y., et al. (2004). Mental health need and access to mental health services by youths involved with child welfare: A national survey. Journal of the American Academy of Child & Adolescent Psychiatry, 43(8), 960-970. Cerezo, M., & Frias, D. (1994). Emotional and cognitive adjustment in abused children. Child Abuse & Neglect, 18(11), 923-932. Costello, E. J., Angold, A., Burns, B. J., Stangl, D. K., Tweed, D. L., Erkanli, A., et al. (1996). The Great Smoky Mountains Study of Youth. Goals, design, methods, and the prevalence of DSM-III-R disorders. Archives of General Psychiatry, 53(12), 1129-1136. Famularo, R., Kinscherff, R., & Fenton, T. (1992). Psychiatric diagnoses of maltreated children: preliminary findings Journal of the American Academy of Child & Adolescent Psychiatry, 31(5), 863-867. ADVANCES IN SOCIAL WORK, Spring 2009, 10(1) 35 Farmer, E. M. Z., Burns, B. J., Chapman, M. V., Phillips, S. D., Angold, A., & Costello, E. J. (2001). Use of mental health services by youth in contact with social services. Social Service Review, 75(4), 605-624. Fedoravicius, N., McMillen, J. C., Rowe, J., Kagotho, N., & Ware, N. C. (2008). Funneling child welfare consumers into and through the mental health system: Assessment, referral, and quality issues. Social Service Review, 82(2), 273-290. Foster, K.A., & Stiffman, A.R. (2009). Child welfare workers’ adoption of decision support technology. The Journal of Technology in Human Services, 27(2), 1-21. Gardner, W., Mulvey, E. P., & Shaw, E. C. (1995). Regression analyses of counts and rates: Poisson, overdispersed Poisson, and negative binomial models. Psychological Bulletin, 118(3), 392-404. Garland, A. F., Hough, R. L., McCabe, K. M., Yeh, M., Wood, P. A., & Aarons, G. A. (2001). Prevalence of psychiatric disorders in youths across five sectors of care. Journal of the American Academy of Child & Adolescent Psychiatry, 40(4), 409-418. Garland, A. F., Landsverk, J. L., Hough, R. L., & Ellis-MacLeod, E. (1996). Type of maltreatment as a predictor of mental health service use for children in foster care. Child Abuse & Neglect, 20(8), 675-688. Glisson, C., & Green, P. (2006). The effects of organizational culture and climate on the access to mental healthcare in child welfare and juvenile justice systems. Administration and Policy in Mental Health and Mental Health Services Research, 33(4), 433-448. Halfon, N., Berkowitz, G., & Klee, L. (1992). Mental health service utilization by children in foster care in California. Pediatrics, 89(9), 1238-1244. Harman, J. S., Childs, G. E., & Kelleher, K. J. (2000). Mental health care utilization and expenditures by children in foster care. Archives of Pediatrics and Adolescent Medicine, 154(11), 1114-1117. Horwitz, S. M., Hoagwood, K., Stiffman, A. R., Summerfeld, T., Weisz, J. R., Costello, E. J., et al. (2001). Reliability of the Services Assessment for Children and Adolescents. Psychiatric Services, 52(8), 1088-1094. Hurlburt, M. S., Leslie, L. K., Landsverk, J., Barth, R. P., Burns, B. J., Gibbons, R. D., et al. (2004). Contextual predictors of mental health service use among children open to child welfare. Archives of General Psychiatry, 61(12), 1217-1224. Kaufman, J. (1991). Depressive disorders in maltreated children. Journal of the American Academy of Child & Adolescent Psychiatry, 30(2), 257-265. Kazdin, A. E., Moser, J., Colbus, D., & Bell, R. (1985). Depressive symptoms among physically abused and psychiatrically disturbed children. Journal of Abnormal Psychology, 94(3), 298-307. Bunger, Stiffman, Foster, Shi/CHILD WELFARE WORKERS’ CONNECTIVITY 36 Leslie, L. K., Gordon, J., N., Lambros, K., Premji, K., Peoples, J., & Gist, K. (2005). Addressing the developmental and mental health needs of young children in foster care. Journal of Developmental and Behavioral Pediatrics, 26(2), 140-152. Leslie, L. K., Hurlburt, M. S., James, S., Landsverk, J., Slymen, D. J., & Zhang, J. (2005). Relationship between entry into child welfare and mental health service use. Psychiatric Services, 56(8), 981-987. Leslie, L. K., Hurlburt, M. S., Landsverk, J., Barth, R., & Slymen, D. J. (2004). Outpatient mental health services for children in foster care: a national perspective. Child Abuse & Neglect, 28(6), 699-714. Leslie, L. K., Landsverk, J. L., Ezzet-Lofstrom, R., Tschann, J., Slymen, D. J., & Garland, A. F. (2000). Children in foster care: factors influencing outpatient mental health service use. Child Abuse & Neglect, 24(4), 465-476. McMillen, J. C., Fedoravicius, N., Rowe, J., Zima, B. T., & Ware, N. (2007). A crisis of credibility: Professionals’ concerns about the psychiatric care provided to clients of the child welfare system. Administration and Policy in Mental Health and Mental Health Services Research, 34(3), 203-212. Mechanic, D. (1979). Correlates of physician utilization: Why do major multivariate studies of physician utilization find trivial psychosocial and organizational effects? Journal of Health and Social Behavior, 20(4), 387-396. Phillips, K. A., Morrison, K. R., Andersen, R., & Aday, L. A. (1998). Understanding the context of healthcare utilization: assessing environmental and provider-related variables in the behavioral model of utilization. Health Services Research, 33(3), 571-596. Ryan, J. P., Garnier, P., Zyphur, M., & Zhai, F. (2005). Investigating the effects of caseworker characteristics in child welfare. Children and Youth Services Review, 28(9), 993-1006. Shaffer, D., Fisher, P., Dulcan, M. K., Daview, M., Piacentini, J., Schwab-Stone, M. E., et al. (1996). The NIMH Diagnostic Interview Schedule for Children Version 2.3 (DISC-2.3): Description, acceptability, prevalence rates, and performance in the MECA study. Journal of the American Academy of Child & Adolescent Psychiatry, 35(7), 865-877. Smith, B. D., & Donovan, S. E. F. (2003). Child welfare practice in organizational and institutional context. Social Service Review, 77, 541-563. Stiffman, A. R., Chen, Y.-W., Elze, D., Doré, P., & Cheng, L. C. (1997). Adolescents' and providers' perspectives on the need for and use of mental health services. Journal of Adolescent Health, 21(5), 335-342. Stiffman, A.R., Foster, K.A., Hamburg, K., & Doré, P. (2003). IMPROVE: A software program to improve assessments and multisector referrals. Paper presented at the Research and Training Center for Children's Mental Health, Tampa, FL. ADVANCES IN SOCIAL WORK, Spring 2009, 10(1) 37 Stiffman, A. R., Freedenthal, S., Doré, P., Ostmann, E., Osborne, V., & Silmere, H. (2006). The role of providers in mental health services offered to American-Indian youths. Psychiatric Services, 57(8), 1185-1191. Stiffman, A. R., Hadley-Ives, E., Doré, P., Polgar, M., Horvath, V. E., Striley, C., et al. (2000). Youths' access to mental health services: The role of providers' training, resource connectivity, and assessment of need. Mental Health Services Research, 2(3), 141-154. Stiffman, A. R., Horwitz, S. M., Hoagwood, K., Compton, W. I., Cottler, L., Bean, D. L., et al. (2000). The Service Assessment for Children and Adolescents (SACA): Adult and child reports. Journal of the American Academy of Child & Adolescent Psychiatry, 39(8), 1032-1039. Stiffman, A. R., Pescosolido, B., & Cabassa, L. J. (2004). Building a model to understand youth service access: The Gateway Provider Model. Mental Health Services Research, 6(4), 189-198. Stiffman, A. R., Striley, C., Horvath, V. E., Hadley-Ives, E., Polgar, M., Elze, D., et al. (2001). Organizational context and provider perception as determinants of mental health service use. The Journal of Behavioral Health Services and Research, 28(2), 188-204. Stroul, B. A., & Friedman, R. M. (1986). A system of care for severely emotionally disturbed children and youth. Washington, DC: Georgetown University Child Development Center, National Technical Assistance Center for Children’s Mental Health. Trupin, E., Tarico, V., Low, B., Jemelka, R., & McClellan, J. (1993). Children on child protective service caseloads: Prevalence and nature of serious emotional disturbance. Child Abuse & Neglect, 17(3), 345-355. U.S. Department of Health and Human Services, Administration on Children, Youth and Families. (2003). Safety, permanency and well-being: Child welfare outcomes 2000: Annual report to Congress. Retrieved June 24, 2008 from http://www.acf.hhs.gov/programs/cb/pubs/cwo03/cwo03.pdf. Uzzi, B., & Lancaster, R. (2003). Relational embeddedness and learning: The case of bank loan managers and their clients. Management Science, 49(4), 383-400. Walrath, C. M., Ybarra, M. L., Sheehan, A. K., Holden, E. W., & Burns, B. J. (2006). Impact of maltreatment on children served in community mental health programs. Journal of Emotional and Behavioral Disorders, 14(3), 143-156. Author’s note: Address correspondence to: Alicia C. Bunger, The Brown School of Social Work, 1 Brookings Drive, Campus Box 1196, St. Louis, MO, 63130. Email: abunger@gwbmail.wustl.edu. http://www.acf.hhs.gov/programs/cb/pubs/cwo03/cwo03.pdf mailto:abunger@gwbmail.wustl.edu Bunger, Stiffman, Foster, Shi/CHILD WELFARE WORKERS’ CONNECTIVITY 38 APPENDIX A: Service Domains and Categories Domain Service Category Health and Education Any Public Health Clinic with Mental Health Services Pregnancy related services or similar resources A school social worker, guidance counselor or school psychologist Special schools Job training resource Educational resources (e.g., tutoring) Inpatient Resources Psychiatric hospital or psychiatric or medical units in a general hospital for emotional or behavioral problems Drug or Alcohol treatment units Residential treatment centers Group or foster homes Detention center/prison or jails Emergency shelters for emotional or behavioral problems Other places like summer treatment programs or boarding schools Outpatient Resources Community mental health center or outpatient mental health clinics Professionals (e.g., psychologist, psychiatrist, social worker or marriage or family counselor) not mentioned Day treatment programs Intensive in-home services Emergency room that treats emotional/behavioral problems Pediatrician/family doctors for emotional or behavioral problems Probation or juvenile corrections services Self-help groups (e.g., AA, 12-step programs, or peer youth counseling program) Other Crisis intervention services (e.g., suicide hotlines) Social services (basic needs) Religious providers of services (e.g., churches, ministers) Life skills programs for children/youth Family or parenting programs Victim’s programs (domestic violence programs; crime victims’ assistance) Any other resources which service the drug/alcohol/mental health needs of children/youth INTRODUCTION