Introduction: (find references) Dialog, 16(2), 75-89 Copyright © 2013, ISSN: 1930-9325 RESEARCH ARTICLE Parent Voices: An Investigation of the Response Process Validity of the Child Behavior Checklist with Head Start Parents Marlo A. Perry University of Pittsburgh Yumiko S. Miller Dallas, TX Rachel A. Fusco University of Pittsburgh John Fantuzzo University of Pennsylvania The purpose of the present study was to examine the response process validity of the Child Behavior Checklist (CBCL; Achenbach, 1991) with Head Start parents. A group of 92 parents was asked to sort the items from the CBCL (Achenbach, 1991) into three piles (Not Comfortable, Comfortable, Very Comfortable) in response to the following question, “to what degree would you feel comfortable answering this question honestly?” Next, parents were asked to re-sort the “Not Comfortable” items into four piles indicating the reason for their discomfort: Need More Information; Don’t Understand; Offensive; Threatening. Overall, there were 29 items that parents most frequently categorized as “Not Comfortable”. The two reasons most frequently given by parents were that they found the questions offensive or threatening. Implications for early childhood education research and practice are discussed. The percentage of young children living in poverty in the U.S. has been steadily increasing in the last decade, with 41% of children under 6 living in low-income families in 2000 rising to 44% in 2008 (Wight & Chau, 2009). Minority children are disproportionately poor, with just over two- thirds of Black, American Indian, and Hispanic children under age 6 living in low-income families, as compared to less than one-third of White children. Research has consistently demonstrated the negative sequelae that poverty and comorbid risks have on young children’s cognitive and social-emotional development (National Research Council, 2002; Shonkoff & Marshall, 2000). 76 PERRY ET AL. Federal initiatives, such as Head Start, have been charged with ameliorating some of the risks associated with poverty and boosting the school readiness skills of low-income children (U.S. Department of Health and Human Services, Administration for Children and Families, 2010). Such initiatives have been scrutinized in recent years (Zigler & Styfco, 2004), and there has been a call for more rigorous evidence-based practice. In fact, the Head Start Reauthorization Act (Head Start Reauthorization Act of 2007) calls for the use of scientifically valid assessments that support classroom instructional practices and program evaluation. Further, it mandates that the measures used be “developmentally, linguistically, and culturally appropriate for the population served” and be “high-quality research-based measures that have been demonstrated to assist with the purposes for which they have been devised” (Head Start Reauthorization Act of 2007, Section 641A). As part of the Head Start Reauthorization (2007), the National Research Council was tasked with conducting a study on the appropriate use and development of early childhood assessments. The resulting report highlights the need for researchers to hold to strict standards when selecting assessment tools. Specifically, researchers must ensure that the tools have strong psychometric properties and are appropriate for different ethnic, language, special needs, and age groups (Snow & Van Hemel, 2008). Additionally, the report draws attention to the potential negative consequences of using measures that are inappropriate for minority populations. Bias may come into play when assessment content is inappropriate for a population due to contextual or cultural differences (Snow & Van Hemel, 2008). Such bias undermines the validity of those assessment tools and their findings. Further, the National Research Council report (Snow & Van Hemel, 2008) invokes a more modern conceptualization of validity, which posits that the unitary concept of construct validity (as opposed to earlier conceptualizations of face, content, and criterion validity) is supported by five sources. These include content, response process, internal structure, relations to other variables, and consequences (Cook & Beckman, 2006; Snow & Van Hemel, 2008). Response process validity is particularly important when thinking about cultural appropriateness of assessment tools, as it reflects the relationship between the intended construct and the thought processes of the test-takers or participants (Cook & Beckman, 2006). Different cultural groups may have different response processes to assessment content, resulting in potentially invalid assessments for those groups. Interviews and focus groups are recommended to examine response process validity of assessment tools (Beckman, Cook, & Mandrekar, 2005; Bornstein, 2011). The Child Behavior Checklist (CBCL; Achenbach, 1991) is a commonly used measure of behavioral adjustment for young children, and has been used in several evaluations of large scale intervention projects, including the Nurse Home Visitation Program and the Comprehensive Child Development Program. However, racial/ethnic differences on CBCL syndromes have been found among community samples, bringing into question the reliability and validity of the CBCL for minority children (LeBoeuf, Fantuzzo, & Lopez, 2010; Sandberg, Meyer-Bahlburg, & Yager, 1991). An item analysis of the CBCL also found racial variability in items responses that suggested that African American parents may have different conceptualizations of problem behaviors than Caucasian parents (LeBoeuf et al., 2010; Ngo, 2007). Further, research has shown that the two behavioral dimensions of the CBCL (Externalizing and Internalizing) most commonly used to make decisions about program effectiveness did not hold up for a community sample of young, low-income children (LeBoeuf, et al., 2010). PARENT VOICES 77 The reasons for the lack of validity of this measure for low-income, minority samples are not yet well understood. LeBoeuf and colleagues (2010) highlight the low prevalence of many of the items in a community sample. This may be due to actual low levels of those behaviors; however, it may also be due to parents not responding honestly to the items. Some studies have shown that parents of low income levels are less accurate in their reporting of developmental problems in their children, in part due to parents skipping questions on written inventories (Glascoe & Dworkin, 1995); the authors speculate that this may be a result of reading difficulties within this population. However, another possibility is that parents were not comfortable answering some types of questions about their children and so instead left them blank. Low- income, minority parents may be particularly wary of endorsing negative items on behavioral assessments of their children, especially if they don’t understand why the information is being collected. Careful questioning is needed in order to collect quality information from parents (Dewey, Crawford, & Kaplan, 2003; Glascoe, Altemeier, & MacLean, 1989; Squires, Bricker, Heo, & Twombly, 2001); additionally, the setting in which concerns are elicited may influence parental report (Glascoe & Dworkin, 1995). For example, parents may feel more comfortable sharing some concerns with their pediatrician rather than their child’s teacher. The overarching purpose of this study was to examine the response process validity of the CBCL with Head Start parents. In other words, we wanted to try to understand the perspectives of Head Start parents when they are asked to respond to items on the CBCL and therefore to understand what contributes to invalid measurement. Since the CBCL is deficit-based, we also included items from another parent-report measure of children’s social-emotional development, the Penn Interactive Peer Play Scale (PIPPS; Fantuzzo, Coolahan et al., 1998; Fantuzzo, Sutton- Smith, Coolahan, Manz, Canning, & Debnam, 1995). Developed in partnership with Head Start parents, the PIPPS asks questions about young children’s behaviors within peer contexts. The reason for including these additional items was simply to provide a more balanced set of questions to which parents could respond. Using a mixed methods approach, the present study has two major research questions. First, which kinds of questions on the CBCL are Head Start parents comfortable and not comfortable answering honestly? Second, what kinds of reasons do parents give for not wanting to answer particular questions honestly? METHOD Participants Participants were recruited from a large, urban school district Head Start program in the Northeast. A team of five parents was hired through the Head Start Policy Council to serve as parent research assistants in the participant recruitment and data collection process. The Head Start Policy Council is comprised of Head Start parents and community members, and is elected by the larger body of Head Start parents each year. The Council is responsible for the direction of the Head Start program, including program design and operation and planning goals and objectives. Participant recruitment was focused on centers that were identified by Policy Council parents; Policy Council parents then served as contact people at centers, where they talked with other parents about the project and posted flyers. The research team (the first three authors and the five parent research assistants) then recruited parent participants by phone and on site. 78 PERRY ET AL. The parent research assistants were provided with education and training about the research process and about the protocol for this particular project. The parent research assistants recruited 92 parents for this project. Participating parents completed a consent form and provided basic demographic information. Most of the respondents were African American females. More than half of the sample was employed full- or part-time, and 48% had some college or vocational education (see Table 1 for more information). TABLE 1 Demographic characteristics of Head Start parent participants (N = 92) Characteristic Percent Relationship to program a Current parent 70 Former parent 33 Current Policy Council representative 27 Participant in Male Involvement group 15 Sex Male 20 Female 80 Relationship to Head Start child Mother 59 Father 11 Step-parent 12 Extended family member 10 Other (e.g. non-family primary caregiver) 8 Race African American 82 Caucasian 6 Hispanic 7 Other 6 Marital status Single 55 Married/cohabitating 28 Divorced/widowed/separated 15 Education level Less than high school 17 High school graduate 23 Some college and/or vocational courses 46 College degree or higher 14 Employment status Full time 36 Part time 21 School or training program 33 Unemployed 10 Note. a Parents could be members of multiple categories (e.g. a current parent and a Policy Council leader). PARENT VOICES 79 Measures Child Behavior Checklist (CBCL). The CBCL consists of 113 items and provides scores on eight subscales: Withdrawn, Somatic Complaints, Anxious/Depressed, Social Problems, Thought Problems, Attention Problems, Delinquent Behavior, and Aggressive Behavior. In addition, scores on three overarching composites can be determined: Total Problems, Internalizing, and Externalizing. The Total Problems composite is comprised of 111 items, and measures overall difficulties in all of the areas assessed by the CBCL. Items are rated on a three-point scale (Very True or Often True, Somewhat or Sometimes True, and Not True). The CBCL was normed in 1989 on a national sample of 2,368 children, who were selected to be nationally representative in terms of ethnicity, SES, geographical region, and urban/suburban/rural residence (Achenbach, 1991). Test authors report solid psychometric properties (see Achenbach, 1991). Penn Interactive Peer Play Scale-Parent version (PIPPS-P). The Penn Interactive Peer Play Scale (PIPPS; Fantuzzo, Coolahan et al., 1998; Fantuzzo, Sutton-Smith, Coolahan, Manz, Canning, & Debnam, 1995) was designed as an assessment of the interactive peer play behavior of young children living in low-income urban areas The PIPPS identifies children who demonstrate successful peer play interactions and those children who experience less successful peer play. The parent version assesses how often (i.e., Never, Seldom, Often, or Always) the play behavior has been observed in the home and neighborhood in the most recent two-month period. The 32-item measure encompasses three underlying dimensions of classroom peer play behaviors: Play Interaction, Play Disruption, and Play Disconnection (Coolahan et al., 2000; Fantuzzo, Mendez et al., 1998; Fantuzzo, Coolahan et al., 1998). Internal consistency for the parent version shows strong reliability for these three factors (r = .74, .84, and .81, respectively. Multimethod, multisource validity analyses further substantiated the PIPPS dimensions for this population of preschool children (Fantuzzo & McWayne, 2002; Fantuzzo, Mendez, & Tighe, 1998; McWayne, Sekino, & Fantuzzo, 2005). Procedures In a one-on-one context, each participant was given informed consent by a member of the research team and was provided with information about the overall goal of the project. Participants were guided through the process (as detailed below) of sorting the assessment items and were then asked more detailed questions about why they categorized items the way that they did. Sessions were held in meeting rooms in several Head Start centers; each individual session took approximately 45 minutes. Parents were given a $20 gift certificate to a local store as a token of appreciation for their time and contribution; they were also reimbursed for travel expenses. Parent participants were asked to sort the 144 CBCL and PIPPS items into three piles (Not Comfortable, Comfortable, Very Comfortable) in response to the following question, “to what degree would you feel comfortable answering this question honestly?” Parents were given further explanation that Very Comfortable meant that they were eager to share that information about their child, Comfortable indicated ease or perhaps ambivalence about sharing that information about their child, and Not Comfortable meant that there were feelings of unease associated with sharing that information about their child. Next, parents were asked to re-sort the 80 PERRY ET AL. “Not Comfortable” items into four piles indicating the reason for their discomfort: Need More Information; Don’t Understand; Offensive; Threatening. Pilot work with small groups of 4-6 parents each (n=34) revealed these overarching categories as the primary reasons why parents were uncomfortable answering particular questions. Finally, a follow-up interview probed parents’ rationale for sorting the items into these four “reason” piles. More specifically, parents were asked to share what it was about that item that made them categorize it that way (for example, they were asked what it was about a particular item that made it seem offensive or what it was about a particular item that they didn’t understand). RESULTS Overall, there were 29 CBCL items that parents most frequently categorized as “Not Comfortable” (see items listed in Table 2). 1 Chi-square analyses were used to determine if there were significant differences in responses based on demographic variables. No differences were found in parent responses for “Not Comfortable” items between groups based on sex, marital status, race, or education level. Next, we wanted to understand why parents rated these particular questions as “Not Comfortable.” Results indicated that parents primarily found these questions to be “Offensive” or “Threatening.” Because of the high frequencies of these two reasons (“Offensive” and/or “Threatening”; see Table 2), the authors focused the subsequent qualitative analyses on only these two reasons. In order to help organize findings overarching content categories were created for the items that parents rated as “Not Comfortable.” The categories were based on the content of each item (i.e., the behavior that was being asked about) and were determined by consensus by three of the authors. These content categories were as follows: antisocial behaviors (8 items), physically aggressive behaviors (2 items), indices of self-harm (3 items), issues related to thought disorders/problems (7 items), sex-related behaviors (6 items), and physical symptoms (3 items). Content categories allowed the authors to see if there were patterns in item content that related to their comfort level in answering such an item honestly. 1 One item from the PIPPS (“Is physically aggressive”) was also frequently categorized as “Not Comfortable”. However, since PIPPS items were included only to provide a more balanced array of items to which parents could respond, and not to provide a comparison to the CBCL, further examination of this item is not included in this paper. PARENT VOICES 81 TABLE 2 Categorization of CBCL items endorsed as “Not Comfortable” by parents (N=92) Item Content Reason for rating “Not Comfortable” More Not Threatening Offensive Info understand CBCL 5 Behaves like the opposite sex 4 4 3 31 6 Has bowel movements outside the toilet 2 2 4 19 15 Cruel to animals 2 2 8 11 16 Cruelty, bullying, or meanness to others 5 1 7 8 18 Deliberately harms self/attempts suicide 6 1 8 13 34 Feels others are out to get him/her 0 5 6 9 39 Hangs around w/ others who get in trouble 5 0 3 12 40 Hears sounds/voices that aren’t there 4 1 7 15 52 Feels too guilty 3 5 7 8 56 Has physical problems w/o known medical cause 4 4 5 10 57 Physically attacks people 5 1 5 11 58 Picks nose, skin, or other parts of body 3 0 2 20 59 Plays with own sex parts in public 2 2 6 32 60 Plays with own sex parts too much 3 2 5 28 67 Runs away from home 4 3 6 13 70 Sees things that aren’t there 4 1 11 11 72 Sets fires 6 2 7 12 73 Sexual problems 2 3 4 29 78 Smears/plays w/ bowel movements 6 3 4 24 81 Steals at home 2 0 9 21 82 Steals outside the home 3 1 7 19 84 Strange behavior 3 6 5 15 85 Has strange ideas 4 2 6 11 89 Suspicious 2 3 12 11 91 Talks about killing self 1 1 8 20 96 Thinks about sex too much 3 2 5 24 105 Uses alcohol/drugs for nonmedical purposes 1 2 9 21 106 Vandalism 3 3 8 13 110 Wishes to be of opposite sex 2 4 1 39 Total for each reason 94 66 178 510 Note: Items have been abbreviated for ease of presentation 82 PERRY ET AL. Coding of “Not Comfortable” items. Three of the authors independently coded each parent’s response to the question, “Why did you find this question Offensive/Threatening?” These authors then reviewed initial coding, and definitions for each code were developed, refined, and agreed upon. Based on these finalized codes and definitions, the raters then independently coded each of the items. The codes for items categorized as Offensive were as follows: Not Age Appropriate – Parent responses stating that described behavior is not appropriate for age level of child; Blaming Parents – Parent responses stating concern about parenting skills and/or parent behavior being called into question; Uncomfortable with Subject Matter – Parent responses stating that they were so uncomfortable with the subject matter of the item (e.g., sexual behaviors) that they would not answer; Labeling Child – Parent responses stating concern that others will make assumptions about child and/or accuse the child if the item is endorsed; Unthinkable Behavior for My Child – Parent responses stating that the values represented in the item are so counter to their own values that they cannot answer; Personal – Private/Family Matter – Parent responses stating that the information the item calls for is appropriate only for the family to know; Personal – Professional Matter – Parent responses stating the information called for is private and would only be appropriate to share with a professional (e.g., doctor, psychologist, etc.); Personal – Context-relevant – Parent responses stating the information called for is personal and would only be appropriate to share if they knew more about why the question was being asked and/or for what the information would be used; More Information – Parent responses indicating feelings of discomfort/being offended by item and wanting more details about what information is being asked about; No Code – Parent responses that do not answer question (“why do you find the question offensive/threatening?”) or responses were too ambiguous to code under another category; Normative/Age Appropriate – Parent responses indicating that they feel offended by the implication that there is something wrong with their child when they consider the behavior being asked about to be normal/expected behavior or age appropriate behavior for their child. The codes for items categorized as Threatening were the same as above with the addition of the following: Fear of Consequences – Parent responses indicating concerns for either parents or the child about negative consequences for answering honestly (e.g., report to child protective servics, etc.). Interrater agreement. The three raters met and reviewed each parental response and recorded interrater agreement. Overall interrater agreement for questions parents labeled Offensive was 90.1%. Interrater agreement for reason categories ranged from 75.4% (Uncomfortable with Subject Matter) to 100.0% (Normative Behavior and Professional Matter). Interrater agreement for content categories under Offensive ranged from 87.9% (sex-related behaviors) to 94.2% (thought disorders/problems). Overall interrater agreement for questions that parents labeled Threatening was 94.0%. Interrater agreement for reason categories ranged from 66.7% (Uncomfortable) to 100.0% (Normative and Private/Family Matter) 2 . Interrater agreement for content categories ranged from 88.9% (self-harm) to 96.3% (thought disorders/problems). Items parents found Offensive. Overall, there were 510 parent responses that indicated that a Not Comfortable item was thought to be Offensive, with the two primary reasons being Private and Personal – Context Relevant. Of the 510 responses, 130 (25%) indicated that 2 Only one item that was found to be Threatening was coded as Uncomfortable and only one item under Threatening was coded as Normative. PARENT VOICES 83 the item asked about something considered to a private family matter. Many of these (38%) asked about sex-related topics (e.g., “Behaves like the opposite sex”, “Plays with own sex parts in public”, “Thinks about sex too much”) or anti-social behaviors (25%; e.g., “Cruel to animals”, “Hangs around with others who get in trouble”, “Sets fires”). Slightly fewer (n=109; 21%) of the Offensive items were deemed so because the parent felt it asked about something personal and wanted to know more about the context in which it was being asked (i.e., what was going to be done with the information, who was going to have access to the information, why the information was sought, etc.). Most of these items also fell into the antisocial (26%) and sex- related categories (25%). Table 3 shows the distribution of reasons parents found each item to be Offensive across each of the content areas. 84 PERRY ET AL. Note: Antisocial items = 15, 39, 67, 72, 81, 82, 105, 106; Physical symptom items = 6, 56, 78; Physically aggressive items = 16, 57; Self-Harm items = 18, 58, 91; Sex related items = 5, 59, 60, 73, 96, 110; Thought disorder items = 34, 40, 52, 70, 84, 85, 89. TABLE 3 Distribution of reasons parents found items to be offensive Reason Category (%) Age Blaming Subject Fear Labeling Unthinkable Private Professional Context More Info Normative No code Antisocial (N= 122) Total 13.9 3.3 0.8 0.8 9.0 7.4 27.0 0.8 22.9 4.1 0 9.8 Physical symptoms (N=53) Total 1.9 7.5 13.2 0 1.9 1.9 22.6 0 28.3 7.5 0 15.1 Physically Aggressive (N=19) Total 5.3 0 5.3 0 5.3 15.8 31.6 0 31.6 0 0 5.3 Self-Harm (N=53) Total 15.1 1.9 7.5 0 1.9 0 22.6 5.7 22.6 0 7.5 15.1 Sex Related (N= 183) Total 14.2 2.2 13.7 0 4.9 4.4 26.8 4.9 14.8 3.3 0.5 10.4 Thought Disorder (N= 80) Total 3.8 0 0 1.3 11.3 5 22.5 6.3 26.3 11.3 1.3 11.3 Overall reason category (N = 510) Total 11.0 2.6 7.5 0.4 6.3 4.9 25.5 3.5 21.4 4.7 1.2 11.2 PARENT VOICES 85 Parents offered a number of reasons as to why they found particular items Offensive. Many responses indicated that the items asked about something that was “personal” and “nobody’s business” and therefore they didn’t want to disclose the requested information. One parent responded, “All are personal, family issues; not shared with outsiders.” Another stated, “Implying something’s going on at home…some things should be dealt with at home…and [it is a] home concern unless parent says [otherwise].” Parent responses also frequently indicated that the item was context relevant; in other words, they may only feel comfortable answering it honestly if they knew more about the purpose of the question and what was going to be done with the information. For example, one parent responded, “Too invasive for someone who doesn’t know to ask; might be comfortable if I knew the individual asking.” Another stated, “There would have to be an incident to ask about this…no context or incident to ask about…” More globally, parent responses often included variations of this parent’s statement, “Why do they need to know? What is the purpose of asking?” Other parent responses indicated that they were concerned about their child being labeled (“Saying my child is crazy” and “Seems like you are asking if my child is a pervert”) or that they thought the question was not asking about age appropriate behaviors (“Not appropriate for small children, more for older children. You don’t want to start accusing kids so early before they start doing things…” and “Why do you want to know? Why ask questions like that if kids are in pre-k? Kids not thinking about stuff like that, so why ask?”). Other items were thought to only be appropriate for medical or psychological professionals to ask (“Only speak with a therapist about this”, “Too personal, may be okay for a counselor”, “Only medical personnel should be asking this question”). Items parents found Threatening. In all, there were 178 parent responses that indicated a Not Comfortable item was thought to be Threatening, with the two primary reasons being Private and Labeling. Of the 178 responses, 35 (20%) indicated that the item asked about something thought to be a private family matter. Many of these items deemed private (34%) asked about anti-social behaviors. A lesser number asked about sex-related topics (23%). Slightly fewer of the items categorized as Threatening were categorized this way because parents were concerned that it would result in their child(ren) being labeled. More than half of these items (56%) were related to thought disorders (e.g., “Feels others are out to get him/her”, “Hears sounds or voices that aren’t there”, “Strange behavior”). Table 4 shows the distribution of reasons parents found each item to be Threatening across each of the content areas. Parents provided a variety of reasons why they thought particular items were Threatening. Parent responses about privacy frequently included statements such as, “No one’s business but mine...” and “question is too personal.” One parent found an item to be threatening because she was worried about her child being labeled: “Suspicious and mistrusting. If tell negative things about child so they document things, and this follows child in records for their life…if child being disobedient child at home, then look for negative behavior at school….then labeled at school.” Additional parents concerned about their child being labeled said, “Mistrusting – child might be treated differently based on answer”, “…calling child crazy”, and “…implying my child is sick in the head.” Other parents were fearful of consequences if they answered the item honestly (“Hidden agenda behind question” and “Would be in therapy – could report to DHS to take child away”) or were concerned that they would be blamed for their child’s behavior (“Why would you want to know? Are you questioning my parenting?” and “Implying that child is not getting needed attention”). 86 PERRY ET AL. TABLE 4 Distribution of reasons parents found items to be threatening Reason Category Age Blaming Subject Fear Labeling Unthinkable Private Professional Context More Info Normative No code Antisocial (N= 57) Total 8.8 3.5 0 10.5 15.8 0 21.1 0 10.5 10.5 0 19.3 Physical symptoms (N=13) Total 0 23.1 0 0 7.7 0 23.1 7.7 15.4 15.4 0 7.7 Physically Aggressive (N=12) Total 0 0 8.3 8.3 8.3 0 16.7 0 16.7 25.0 0 16.7 Self-Harm (N=18) Total 0 0 0 11.1 16.7 0 22.2 0 22.2 16.7 0 11.1 Sex Related (N= 24) Total 20.8 0 0 12.5 0 4.2 33.3 0 16.7 0 0 12.5 Thought Disorder (N= 54) Total 0 5.6 0 5.6 33.3 1.9 11.1 0 13.0 9.3 1.9 18.5 Overall reason category (N=178) Total 5.6 4.5 0.6 8.4 18.0 1.1 19.7 0.6 14.0 10.7 0.6 16.3 Note: Antisocial items = 15, 39, 67, 72, 81, 82, 105, 106; Physical symptom items = 6, 56, 78; Physically aggressive items = 16, 57; Self-Harm items = 18, 58, 91; Sex related items = 5, 59, 60, 73, 96, 110; Thought disorder items = 34, 40, 52, 70, 84, 85, 89. PARENT VOICES 87 DISCUSSION This study examined the response process validity of the CBCL with a sample of Head Start parents. The CBCL was developed with a clinical focus to identify behavioral difficulties in children and youth, and has frequently been used in research with Head Start populations. We wanted to better understand why the CBCL has been found to be invalid with low-income, minority populations and asked a sample of Head Start parents to respond to the content of each of the CBCL items in terms of how comfortable they would be answering each item honestly. Overall, this sample of Head Start parents indicated not being comfortable answering 29 items from the CBCL. When asked to consider why they were “not comfortable,” the two reasons most frequently given by parents were that they found the questions offensive or threatening. Among items parents indicated as offensive, the greatest number of parent responses was for sex-related items. For items parents considered threatening, the category with the most parent responses was for items focused on antisocial behaviors. Reasons for finding the items offensive and/or threatening were most frequently because the item asked about personal/private information, they didn’t know enough about the context in which the question was being asked, and they were concerned about their child being labeled if they did answer the question honestly. The present study provides evidence for the lack of response process validity of the CBCL with low-income, minority parents, and as such, provides more information as to why previous studies have found the CBCL to be invalid with low-income, minority samples of young children. LeBoeuf and colleagues (2010) found that the factor structure of the CBCL did not hold up with a community sample of low-income, ethnically diverse preschool children. Other studies have also found that the norms and factor structure of the CBCL were not valid with community samples (Sandberg, Meyer-Bahlburg, & Yager, 1991) or with African- American samples (Lambert, Rowan, Lyubansky, & Russ, 2002). Item analysis of the CBCL has also shown racial variability in item responses, suggesting that African American families may conceptualize problem behaviors differently than Caucasian parents (Ngo, 2007). Findings from the current study illustrate parents’ discomfort with many items on the CBCL and their subsequent reluctance to answer those items honestly (or at all), even if the behavior were true of their child. This can be of particular importance when researchers want to implement measures with poor, minority populations. African-American mothers may be more sensitive about potentially presenting her child in (what she perceives to be) a harsh or even pathological light. African- Americans are generally wary of the motives and intentions of researchers (Corbie-Smith, Thomas, & St. George, 2002; Moseley, Freed, Bullard, & Goold, 2007). Further, African- American children (particularly boys) and their parents are often targeted by educational systems as problematic and deficient (Gilliam, 2005; Ogbu, 1990), so it is understandable that these parents may be on the defensive and may skip items on inventories or may not answer them honestly if they fear that honest answers may result in negative consequences for them or their child. The present study was conducted with a small, convenience sample of primarily African American parents whose children were enrolled in a Head Start program in a large, northeastern city. In order to more comprehensively capture parents’ attitudes about behavioral report items, future research should include expanded samples, including more ethnically diverse sets of parents. Low-income parents from urban, suburban, and rural settings may also have differing views on which kinds of questions they are comfortable answering honestly about their 88 PERRY ET AL. children’s social emotional development. Differences may also be found among sub-groups of low-income parents. For example, adolescent mothers or single mothers or more socially isolated mothers may be more defensive and/or resistant and therefore respond even more negatively to assessment tools like the CBCL. Additionally, future work could elicit more detailed information from parents, either by using a different categorization system or by utilizing more nuanced qualitative research methods. For example, focus groups and/or semi-structured interviews may reveal more information about parents’ responses to particular types of questions about their children. The present investigation has important implications for early childhood education programs. Such programs need to carefully consider the measures they use with parents and make sure they are reliable and valid with the populations they serve. Further, programs need to have a system in place to understand parents’ perspectives in completing these measures. This may be through something like Head Start’s Parent Policy Council, or perhaps a more informal committee of parents. If parents find assessment items offensive or threatening and therefore don’t answer them honestly or leave them blank, that could have negative repercussions for programs, as well as parents. It could result in an underreporting of problems, which could then result in inadequate services or resources. There is also the additional risk for alienating parents from the program because they are being asked these kinds of questions, with no understanding of what information is sought and why it is sought. On a larger scale, if these measures are being used to evaluate the overall effectiveness of a program, the findings could be rendered invalid, thereby jeopardizing the entire program. Parents are an invaluable source from whom to gather information about a child’s development and functioning. They provide a unique perspective that can complement or supplement perspectives of teachers and other individuals with whom children have regular contact. In order to truly capture parents’ observations and beliefs about their children, however, programs need to have a culturally sensitive approach that carefully considers the respondents’ perspectives and contexts. REFERENCES Achenbach, T. M. (1991). Manual for the Child Behavior Checklist/4-18 and 1991 Profile. Burlington, VT: Department of Psychiatry, University of Vermont. Beckman, T. J., Cook, D. A., & Mandrekar, J. N. (2005). What is the validity evidence for assessments of clinical teaching? Journal of General Internal Medicine, 20, 1159-1164. Bornstein, R. F. (2011). Toward a process-focused model of test score validity: Improving psychological assessment in science and practice. Psychological Assessment, 23, 532-544. Cook, D. A., & Beckman, T. J. (2006). Current concepts in validity and reliability for psychometric instruments: Theory and application. The American Journal of Medicine, 119, 166.e7-166.e16. Coolahan, K. C., Fantuzzo, J., Mendez, J., & McDermott, P. (2000). Preschool peer interactions and readiness to learn: Relationships between classroom peer play and learning behaviors and conduct. Journal of Educational Psychology, 92, 458-465. Corbie-Smith, G., Thomas, S. B., & St. George, D. M. M. (2002). Distrust, race, and research. Archives of Internal Medicine, 162, 2458-2462. Dewey, D., Crawford, S. G., & Kaplan, B.J. (2003). Clinical importance of parent ratings of everyday cognitive abilities in children with learning and attention problems. Journal of Learning Disabilities, 36 (1), 87-95. PARENT VOICES 89 Fantuzzo, J., & McWayne, C. (2002). The relationship between peer-play interactions in the family context and dimensions of school readiness for low-income preschool children. Journal of Educational Psychology, 94, 79-87. Fantuzzo, J., Coolahan, K., Mendez, J., McDermott, P., & Sutton-Smith, B. (1998). Contextually-relevant validation of peer play constructs with African American Head Start children: Penn Interactive Peer Play Scale. Early Childhood Research Quarterly, 13, 411-431. Fantuzzo, J., Mendez, J., & Tighe, E. (1998). Parental assessment of peer play: Development and validation of the parent version of the Penn Interactive Peer Play Scale. Early Childhood Research Quarterly, 13 (4), 659- 676. Fantuzzo, J., Sutton-Smith, B., Coolahan, K., Manz, P., Canning, S., & Debnam, D. (1995). Assessment of preschool play interaction behaviors in young low-income children: Penn Interactive Peer Play Scale. Early Childhood Research Quarterly, 10, 105-120. Gilliam, W. S. (2005). Prekindergarteners left behind: Expulsion rates in state prekindergarten systems. Foundation for Child Development Policy Brief Series No. 3. New York: Foundation for Child Development. Glascoe, F. P., Altemeier, W. A., & MacLean, W. E. (1989). The importance of parents’ concerns about their child’s development. American Journal of Diseases of Children, 143, 955-958. Glascoe, F., & Dworkin, P.H. (1995). The role of parents in the detection of developmental and behavioral problems. Pediatrics, 95(6), 829-836. Head Start Reauthorization Act of 2007, P.L. 110-134, ACF-IM-HS-08-01 (2007). Lambert, M. C., Rowan, G. T., Lyubansky, M., & Russ, C. M. (2002). Do problems of clinic-referred African- American children overlap with the Child Behavior Checklist? Journal of Child and Family Studies, 11 (3), 271-285. LeBoeuf, W. A., Fantuzzo, J. W., & Lopez, M. L. (2010). Measurement and population miss-fits: A case study on the importance of using appropriate measures to evaluate early childhood interventions. Applied Developmental Science, 14 (1), 45-53. McWayne, C., Sekino, Y., & Fantuzzo, J. (2005). The validity of Head Start teacher assistant report of children’s peer play competencies. NHSA Dialog, 8, 103-120. Moseley, K. L., Freed, G. L., Bullard, C. M., & Goold, S. D. (2007). Measuring African-American parents’ cultural mistrust while in a healthcare setting: A pilot study. Journal of the National Medical Association, 99 (1), 15-21. National Research Council. (2002). Neurons to neighborhoods. Washington, DC: Author. Ngo, V. K. (2007). Contextual analysis of measurement bias in the Child Behavior Checklist for African American children (Doctoral dissertation, Vanderbilt University, 2007). Dissertation Abstracts International, 67 (10- B), 6070. Ogbu, J. U. (1990). Minority education in perspective. Journal of Negro Education, 59, 45-57. Sandberg, D. E., Meyer-Bahlburg, H. F., & Yager, T. J. (1991). The Child Behavior Checklist nonclinical standardization samples: Should they be utilized as norms? Journal of the American Academy of Child and Adolescent Psychiatry, 30 (1), 124-134. Shonkoff, J. P., & Marshall, P. C. (2000). The biology of developmental vulnerability. In J. P. Shonkoff, & S. J. Meisels (Eds.), Handbook of early childhood intervention (2nd ed., pp. 35–53). Cambridge, UK: Cambridge University Press. Snow, C. E., & Van Hemel, S. B. (2008). Early childhood assessment: Why, what, and how? National Research Council of the National Academies Report. Washington, DC: The National Academies Press. Squires, J., Bricker, D. D., Heo, K., & Twombly, E. (2001). Identification of social-emotional problems in young children using a parent-completed screening measure. Early Childhood Research Quarterly, 16, 405-419. U.S. Department of Health and Human Services, Administration for Children and Families. (January 2010). Head Start Impact Study. Final Report. Washington, DC. Wight, V. R., & Chau, M. (2009). Basic facts about low-income children, 2008: Children under age 6. New York: National Center for Children in Poverty. Zigler, E. & Styfco, S. J. (2004). The Head Start Debates. Baltimore, MD: Paul H. Brookes Publishing Co.