Dialog, 16(3), 172-177 Copyright © 2013, ISSN: 1930-9325 Lillian K. Durán, Special Education, Utah State University; Mark S. Innocenti, Center for Persons with Disabilities, Utah State University; M. Brooke Robertshaw, Instructional Technology, Utah State University. Correspondence concerning this article should be addressed to Lillian K. Durán, Department of Special Education and Rehabilitation, Utah State University, 2865 Old Main Hill, Logan, UT 84322-2865. E-mail: Lillian.duran@usu.edu RESEARCH-TO-PRACTICE SUMMARY Using the Bilingual Early Language Assessment: Strengths and Limitations Lillian K. Durán, Mark S. Innocenti, and M. Brooke Robertshaw Utah State University The Bilingual Early Language Assessment (BELA) was developed by the Harvard Graduate School of Education for the Cambridge Public School District in 2002 to respond to a growing need for an assessment in multiple languages. The BELA is available in 11 different languages and can be downloaded at no cost. However there is no psychometric data available on the BELA to inform end users regarding the quality of the measure. This paper reports on a pilot study investigating the psychometric properties of the English and Spanish BELA including the internal consistency, test-retest reliability, and concurrent validity of the BELA with the Preschool Language Scale-4. The population of dual language learners (DLLs) in Head Start has dramatically risen over the last 10 years. The complexity of providing culturally and linguistically responsive services to this growing population in Head Start is outlined in the report “Dual Language Learning: What does it take?” (U.S. Department of Health and Human Services, 2008). The report describes the findings of a national assessment of Head Start programs that included focus groups, conference calls, and multiple meetings with individuals within the Head Start community to determine program needs for serving DLLs. A major finding involves the need for better assessments. Local programs are required to conduct developmental screenings and ongoing assessments of enrolled children. Yet, there exist few valid and reliable assessment instruments for evaluating progress in language and literacy development, as well as development in other domains of learning for children who are learning two languages. Without accurate assessment information, staff are not able to properly support the child’s development, identify progress, individualize the curriculum fully, or identify behavior or delays requiring further evaluation and possible intervention. (U.S. Department of Health and Human Services, 2008, p. 5) The focus of this paper is on one measure that is currently available in 11 different languages. The Bilingual Early Language Assessment (BELA; Tabors & Heise-Bagorria, 2004) was designed for use with 2.9 to 5 year olds for the Cambridge, MA, public schools. It provides a USING THE BELA 173 general measure of ability in the child’s home language and English on language and conceptual skills. It is also designed as a progress monitoring tool that can be administered several times over the school year. The BELA can help practitioners understand what concepts children have in their home language and in English because it taps the exact same skills in each language. The test is not a screening tool or a formal language proficiency measure. Instead, it is meant to provide information to guide instruction. Little information is available regarding the development of the instrument, however the measure was presented at the National Dual Language Head Start Institute in 2008 (Heise-Bagorria, 2008). The test is available at no cost online; however, there are materials that need to be purchased in order to administer the instrument (www.cpsd.us/BELA). The BELA is currently available in Arabic, Bangla, Chinese, English, Haitian-Creole, Portuguese, and Spanish. The Minnesota Department of Education has also recently translated the tool into Hmong, Oromo, Russian, and Somali and these versions are also available online at (http://education.state.mn.us/MDE/Learning_Support/Early_Learning_Services/Early_Childhoo d_Programs/Help_Me_Grow_Prog_Serv/Administration/index.html). The authors of the BELA explicitly encourage translation of the BELA into other languages that can then be posted on their website. It is currently considered best practice to measure bilingual children in each of their languages to more accurately describe their abilities across their languages (National Association for the Education of Young Children [NAEYC], 2005; Paradis, Genesee, & Crago, 2010; Peña & Halle, 2011). A child’s overall abilities may be underestimated if only assessed in one of their languages (Bedore, Peña, García, & Cortez, 2005; Boyce, Gilliam, Innocenti, Cook, & Ortiz, in press; Peña & Kester, 2004). Children who speak more than one language will have skills distributed across all of their languages based on the context in which they use that language and the quality and quantity of input they have had in each (Hammer, Miccio, & Rodriguez, 2004). In order to accomplish this however, valid and reliable assessment tools must be available in each of a child’s languages to accurately measure his/her overall language ability. The purpose of this paper is to provide preliminary psychometric data on the Spanish and English versions of the BELA (Tabors & Heise-Bagorria, 2004). The vast majority (86%) of DLLs in Head Start speak Spanish. Identifying a psychometrically valid tool that could provide basic information to teachers about a child’s ability in both languages to guide instruction and that could be used across the year to monitor the child’s progress in each language would be a valuable addition to the measures currently available. The BELA is starting to be used in Head Start programs (e.g., Minnesota) while little is known about its psychometric properties. This paper addresses three specific questions regarding the psychometric properties of the BELA. 1. What is the internal consistency of the Spanish and English BELA? 2. What is the test-retest reliability of the Spanish and English BELA? 3. What is the concurrent validity of the BELA with the Preschool Language Scale-4 (PLS-4; Zimmerman, Steiner, & Pond, 2002) in English and Spanish? http://www.cpsd.us/BELA 174 DURAN ET AL. SUMMARY OF RESEARCH METHODS Participants Thirty-nine preschool children (ages 40 to 63 months; M = 50.79; female: n = 17) in four classrooms, attending one Migrant Head Start school located in a rural community in the intermountain region of the U.S. participated in the study. Based on data gathered on the family language questionnaire, children were categorized into three broad language proficiency groups: English dominant (ED, n = 6); Spanish dominant (SD, n = 21); and simultaneously bilingual (SB, n = 9). Procedures Thirty-nine Spanish-speaking preschoolers were administered the BELA (Tabors & Heise- Bagorria, 2004) and the Auditory Comprehension (AC) and Expressive Communication (EC) subscales of the PLS-4 (PLS-4; Zimmerman et al., 2002) in English and Spanish. The BELA was administered twice in each language to each child in November and then again in mid-December or early January due to the holiday break. The PLS-4 was administered once in English and Spanish in mid-December or January. Description of the BELA The BELA includes 10 receptive and 9 expressive items. The items on the receptive subtest include tasks requiring the child to identify common objects, colors, size, quantity, shapes, and actions. On the expressive subtest children are asked to: 1. Provide basic personal information such as their name, age, favorite color and play activity. 2. Repeat sounds, phrases and sentences. 3. Label colors, body parts, common objects, and actions in response to stimulus items. 4. Rote count to 10. Overall, the items on the BELA represent a sampling of tasks found on many popular early language assessments commonly used in the field. The Spanish version is a direct translation of the English version and most items are exactly the same on the Spanish and English versions with the exception of the items requiring the child to repeat sounds, phrases and sentences. No published information could be found regarding the procedures followed in developing the Spanish version. USING THE BELA 175 SUMMARY OF RESULTS Internal Consistency of the BELA Internal consistency is a measure of how well items on a test are correlated to one another. A good measure has strong correlations between items on the test particularly within subsections of the test as this indicates that the items are tapping the same construct. The items on the English receptive and expressive sections showed acceptable levels of internal consistency, however the Spanish version did not demonstrate good internal consistency. This could mean that the items in Spanish do not necessarily function as well in measuring receptive and expressive language. Test-Retest Reliability of the BELA Test-retest reliability is important because it helps us to know if we give the test to a child again whether or not we are likely to get similar results. Test-retest reliability was calculated at the subtest level. The English receptive and expressive subtests achieved higher levels of test-retest reliability than did the Spanish version. However both the English and Spanish version had acceptable levels of test-retest reliability. Concurrent Validity Concurrent validity is a form of validity that compares a test against a benchmark test to examine whether the two tests are measuring the same construct (Nunnally & Bernstein, 1994). Each child’s performance on the PLS-4 to the BELA was compared on both the expressive and receptive subtests. We found that performance on English BELA was correlated with performance on the English PLS-4 and performance on the Spanish BELA was correlated with performance on the Spanish PLS-4. The PLS-4 is a widely used language assessment tool that includes a large standardization sample and the Spanish version includes Spanish-English bilingual preschool children in the U.S. It is recognized as one of the higher quality language assessment tools available for use with English and Spanish speaking preschoolers in the U.S. DISCUSSION AND IMPLICATIONS FOR PRACTICE The primary objective of this study was to provide pilot data on the psychometric properties of the BELA (Tabors & Heise-Bagorria, 2004). We specifically investigated the internal consistency, test-retest reliability, and the concurrent validity of the BELA with the PLS-4. However, given the small sample size included in this study these data can only provide preliminary evidence regarding the validity and reliability of the Spanish and English versions of the BELA with Spanish-speaking preschoolers in Head Start. Based on the data gathered implications for using the BELA in practice are as follows. 1. The BELA should not be used diagnostically as it is not standardized and the results cannot inform decisions about whether or not a child has a language delay. 176 DURAN ET AL. 2. The BELA can be used to provide more information about what specific skills children have in each of their languages. The BELA includes items that tap knowledge about colors, shapes, size, and numbers. Additionally, the BELA includes items that probe language skills beyond picture naming such as repeating sounds and phrases and answering open-ended questions such as, “What do you like to do?” Measuring children’s performance in these broader areas could provide more information about the child’s language abilities that relate to classroom communication and performance. Gathering this information is important to facilitate better instructional decision-making. 3. The BELA is also recommended as a progress monitoring tool and teachers could use the BELA to track children’s progress in both of their languages over the school year. This would help teachers to know whether or not a child is making progress and in which language(s). 4. When testing bilingual children to gather information about their language ability in each language it is important to test each language separately. This practice will elicit their best performance in each language rather than switching back and forth between languages. It is also best practice to have one person give the test in one language and another person give the test in the other language. Individuals administering the test should have native-like mastery of the language they are testing. 5. Teachers should consider administering the BELA over the school year to monitor children’s progress in each of their languages. In this way teachers can note the progress children are making and also note areas that need to be targeted in instruction in each language. For example the BELA includes color naming, counting, and shape naming. Teachers can notice whether or not children are acquiring these pre-academic skills in both of their languages. CONCLUSION The BELA was designed for instructional planning and progress monitoring. Given that the instrument is currently available at no cost online, it is important that practitioners have access to pilot data to be aware of the limitations of the instrument. The findings reported here suggest that caution should be taken when using the BELA since questions remain about the reliability of the BELA, especially the Spanish version. However, the BELA has good concurrent validity with the English and Spanish versions of PLS-4, which indicates that it has the potential to provide valid data regarding a child’s language abilities. Replication of this study with a larger sample size is needed to more accurately assess the validity and reliability in the hope of providing definitive recommendations regarding the appropriate uses of the BELA for practitioners. However, given the dearth of assessments available in languages other than English, the BELA at least provides teachers with a systematic and relatively easy way to collect information about children’s language abilities in their home language and English; an important contribution to the field. USING THE BELA 177 REFERENCES Bedore, L. M., Peña, E. D., García, M., & Cortez, C. (2005). Conceptual versus monolingual scoring: When does it make a difference? Language, Speech, and Hearing Services in the Schools, 36, 188-200. Boyce, L.K., Gillam, S.L., Innocenti, M.S., Cook, G.A., & Ortiz, E. (in press). An examination of language input and vocabulary development of young Latino dual language learners living in poverty. First Language. Hammer, C., Miccio, A.W., & Rodriguez, B.L. (2004). Bilingual language acquisition and the child socialization process. In B. A. Goldstein (Ed.), Bilingual language development and disorders in Spanish-English speakers (pp. 21- 50). Baltimore, MD: Brookes. Heise-Bagorria, C. (2008, October). The Bilingual Early Language Assessment. Paper presented in symposium at the meeting of the National Head Start Association, Washington DC. National Association for the Education of Young Children. (2005). Screening and assessment of young English- language learners: Supplement to the NAEYC and NAECS/SDE joint position statement on early childhood curriculum, assessment, and program evaluation. Retrieved from http://www.naeyc.org/files/naeyc/file/positions/ELL_Supplement_ Shorter_Version.pdf Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill. Paradis, J., Genesee, F., & Crago, M. B. (2010). Dual language development and disorders: A handbook on bilingualism and second language learning. Baltimore, MD: Brookes. Peña, E. D., & Halle, T. G. (2011). Assessing preschool dual language learners: Traveling a multiforked road. Child Development Perspectives, 5(1), 28-32. Peña, E., & Kester, E. S. (2004). Semantic development in Spanish-English bilinguals: Theory, assessment, and intervention. In B. A. Goldstein (Ed.), Bilingual language development and disorders in Spanish-English speakers (pp. 187-212). Baltimore, MD: Brookes. Tabors. P. O., & Heise-Baigorria, C. (2004). Bilingual Early Language Assessment. Retrieved from www.cpsd.us/BELA U.S. Department of Health and Human Services, Administration for Children and Families, Office of Head Start. (2008). Dual language learning: What does it take? Retrieved from http://eclkc.ohs.acf.hhs.gov/hslc/ecdh/eecd/Individualization/Learning%20in%20Two% 20Languages/DLANA_final_2009%5B1%5D.pdf Zimmerman, I. L., Steiner, V., & Pond, R. E. (2002). Preschool Language Scale-4. Birmingham, AL: Psych Corp.