Dialog, 23(1), Copyright © 2020, ISSN: 1930-9325 Implementing Systematic Informal Assessment in Early Education Settings Audra I. Classen The University of Southern Mississippi Jean Kang The University of North Carolina at Greensboro Gregory A. Cheatham The University of Kansas The purpose of this study was to investigate early educators’ current practices, knowledge, beliefs and training needs for utilizing various informal assessment methods. Early educators working Head Start centers and other child care from two southeastern states completed an online survey. The results indicated that participants primarily used one – two informal assessment practices; with anecdotal notes and event/frequency being the most popular. In addition, Head Start teachers used significantly fewer informal assessment methods compared to those in other child care settings. Keywords: informal assessment methods; Head Start; early education Both early childhood (EC) and early childhood special education (ECSE) fields advocate for high quality early education (Catalina & Meyer, 2015; Coppel & Bredekamp, 2009; NAEYC & NAECS/SDE, 2003) that includes informal and formal assessment. “Assessment informs intervention” and is critical to providing quality instruction and services to young children with developmental delays (DEC, 2014, p. 8). National organizations also recommend using multiple assessment methods and emphasize the importance of monitoring children's progress frequently to make database decisions (NAEYC & NAECS/SDE, 2003). Similarly, Head Start and other ECSE programs urge authentic assessments using systematic observation in natural routines and activities (Susman-Stillman, Bailey, & Webb, 2014). For this study, systematic informal assessments (SIAs) is defined as precise observation methods with structured procedures for collecting data about an individual child during natural daily activities, such as snack, free play, centers, recess, and/or transitions. This study specifically focused on early educators’ use of anecdotal notes, event/frequency, partial interval, whole RESEARCH TO PRACTICE IMPLEMENTING SYSTEMATIC INFORMAL ASSESSMENT 41 interval, momentary, duration, and latency sampling. These SIAs were chosen because they are flexible enough to provide a complete view of children’s skills, behaviors, interests, and preferences that can be used to design or revise interventions, make enhancements to a program, and supplement diagnostic evaluations (Bognato, 2005; Wortham & Hardin, 2016). Even though research evidences support importance of using SIA to monitor child progress and program impact, SIAs are used infrequently and narrowly (Neisworth & Bognato, 2004). Previous research reports several barriers to early educators’ use of SIAs. First, early educators have reported limited confidence in selecting SIAs (Krasch & Carter, 2009). Similar to another study reporting that inconsistent use of SIAs may have been due to early educators’ limited knowledge of SIAs and limited administrative support for implementing SIAs (Susman-Stillman et al., 2014). Other barriers to implementing SIAs was lack of time to implement and investigate evidence-based assessment (Banerjee & Luckner, 2013) and difficulties in implementing assessment in the daily routines (Bognato, Neisworth, & Pretti-Frontczak, 2010; Susman-Stillman et al., 2014). Conversely, there are studies reporting that professional development (PD)on SIAs improved early educators understanding and was linked to positive child outcomes (Early et al., 2007; Waitoller & Artiles, 2013). Using this research along with other related research evidences, we have three suggestions for PD on SIAs. The first goal of PD should be to help early educators understand the benefits of using SIA in their daily practice. Second, improve the early educator’s ability to implement SIAs. Finally, enhancing early educators’ self-efficacy related to SIAs (Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005). SUMMARY OF RESEARCH METHODS Survey Instrument and Participants The survey contained seven demographic questions in the first section and twenty-three questions about early educators’ informal assessment practices and training needs in the other section. The total number of 151 EEs responded to the survey. In state one, 88 early educators completed the survey. Of these 88 participants 59% were Caucasian, 37.5% were African American, 1.2% reported two or more races, 1% Asian, .9% other races, and .4% Native North Americans. In state two, 63 early educators completed the survey. Of these 63 participants 71.2% Caucasians, 22.1% African American, 1.6% American Indian, 2.8% Asian, and 9.1% Hispanic or Latino. However, thirty-seven total participant surveys were excluded because the participants failed to answer a significant portion of the informal assessment section (i.e., they only answered two questions in this section). Thus, final analysis included 114 participants. Procedures After downloading survey responses, each researcher first reviewed, cleaned, and assigned a code for each participant from the state to protect confidentiality. Then the two data sets from each state were merged. A descriptive analysis was conducted to gather frequency, percentages, means, and standard deviations. Then independent samples t-tests were conducted to determine if participants’ education setting or years of experience influenced the informal assessment methods used, early educators rating of own knowledge and effectiveness, and training needs. 42 CLASSEN, KANG & CHEATHAM SUMMARY OF RESULTS Current Use of Informal Assessment Methods The four most frequently used SIAs were anecdotal notes (n=100, 87.7%), event/frequency (n=35, 30.7%), other methods (n=20, 17.5%), and partial-interval sampling (n=13, 11.4%). Eight participants (7.0%) reported not using any SIAs. The SIAs used the least were duration (n=8, 7.0%), latency (n=8, 7.0%), momentary time (n =7, 6.1%), and whole interval (n=5, 4.3) sampling. EEs reported that very few administrators required using momentary time (n=6, 5.2%), latency (n=4, 3.5%), whole interval (n=3, 2.6%), duration (n=3, 2.6%), and partial-interval (n=2, 1.7%) sampling. When specific SIAs were not required, early educators selected informal assessment methods based on IEP goals, SIAs previous teachers used, ease of use in daily routines, comfort level of using the SIA, and the possibility of para educators could use the SIA. Educators’ Self-Assessment of their Informal Assessment Knowledge Early educators rated their knowledge of anecdotal notes higher (4.39, SD=0.546) than other informal assessment methods. For example, the average self-rating of knowledge for latency was 2.02 (SD=1.01) while the average rating of event frequency was 2.69 (SD=1.29). The number of participants rating themselves as having a practicing to expert knowledge for anecdotal notes was 102 (97.1%) and for event frequency was 34 (32.4%). Less than 17 participants (16.2%-8.6%) reported a practicing to expert knowledge in the remaining SIAs (i.e., partial, whole, momentary, duration, and latency sampling). EEs working in the Head Start centers had the lowest self- assessment about SIA knowledge with mean scores ranging between 1.67 (i.e., latency) to 1.89 (i.e., partial interval) out of 5. However, this rating was not significantly different statistically when compared to early educators working in the other child care settings. A statistically significant difference between educators from Head Start versus those in other child care settings was found in self- rating of the effectiveness of event/frequency, partial, whole, duration, and latency sampling. Training Needs Identified by Early Educators Early educators reported that the largest barrier to using informal assessment methods was limited training (n=64, 56.1%). Specifically identifying ways, they learned about informal assessment methods, 61 participants (53.5%) reported through in-service training, followed by university course work (n=40; 35.1%), mentoring on the job (n=34; 29.8%) and finally many reported being self-taught (n=29; 25.4%). When asked what level of PD training participants would need in order to implement SIAs comfortably, 52 (45.6%) of the early educator participants reported they would need some to an extreme amount of PD training. IMPLEMENTING SYSTEMATIC INFORMAL ASSESSMENT 43 DISCUSSION FOR IMPLICATIONS FOR PRACTICE Systematic informal assessments (SIAs) that are conducted regularly can aid the early educator in making instructional decisions. Using ongoing assessment, to plan for intentional instruction and monitor children’s progress, is an essential quality indicator for programs serving all young children, including children with disabilities (DEC, 2014; NAEYC & NAECS/SDE, 2003). The results of this study suggest that EEs’ limited informal assessment knowledge impacted their varied use of SIAs. Reports few data collection methods being used in classrooms suggest that EEs may not be adequately prepared to choose multiple data sources for planning and making decisions about instruction and the appropriate services needed (DEC, 2014; Wolraich, Gurwitch, Bruder, & Knight, 2005). Importantly, this study supports other research stating that “teachers’ buy-in” related to using a specific instructional approach influences their frequency and quality of use of the instructional practice learned during the PD (Fixsen et al., 2005). Similarly, PD that supports educators’ beliefs and builds their confidence will improve and sustain learning outcomes long term (Wilkins, 2008). The current results indicate EEs’ beliefs about the effectiveness of each informal assessment method impacted their current practice and use. For example, the top reported SIAs used were anecdotal notes and event frequency and EEs rated anecdotal notes as the most effective and event frequency as somewhat effective. Conversely, the least used SIA of latency sampling was rated as somewhat ineffective. Implications for Practice Several limitations existed in this study and results should be interpreted with caution due to the small sample size and the convenient, snowball sampling method used. Furthermore, participants in the study may not be comparable to the national EC workforce. However, there are important implications for preservice and inservice SIA training that can be gleaned from this study. First, PD should be provided to help educators better understand the benefits and links between assessment and effective curriculum and instruction planning, implementation, and program evaluation. Based on participant responses indicating most of their SIA training occurred during inservice PD, this study signifies a need for pre-service teacher education programs to analyze the content and the approaches used in their assessment courses. Specifically, looking for opportunities to improve EEs ability to implement SIAs. Likewise, other research (Banerjee & Luckner, 2013; DEC, 2014; NAEYC & NAECS/SDE, 2003) suggests that teacher education programs should provide multiple practice opportunities and activities for teacher candidates to use formal and informal assessments in diverse settings, with diverse children, and for multiple purposes (e.g., planning instruction, progress monitoring, and writing meaningful IEP goals). Early educators also reported needing additional training for all SIAs except for anecdotal notes. The barriers to their SIA knowledge was linked to having limited experience selecting appropriate SIAs, not being comfortable using SIAs, and not having preferences for SIA. Thus, we need to enhance EEs’ self-efficacy related to SIAs. PD related to selecting SIA methods and knowing each SIAs time commitment may improve EEs overall use of these SIA methods. In addition, participants in this study reported preferences for online modules and coaching support. These preferences align with other research reporting similar PD preferences (e.g., Howes, James, 44 CLASSEN, KANG & CHEATHAM & Ritchie, 2003; Ramey & Ramey, 2005; Susman-Stillman et al., 2014). Using these findings and supporting PD research a recommended PD checklist was provided in the research article (Table 1). Implementation science (Fixsen et al., 2005) provides a framework for integrating the previously mentioned PD assessment needs and delivery methods. Specifically, the implementation stages can become a checklist for teacher education programs and PD providers wishing to improve their current programs (see Table 1). The checklist for improving informal assessment practices may assist teacher education programs and PD providers in conducting better informal assessment training. Systematic Informal Assessment Resources Early educators who relate to the participants in this study and want to learn more about SIA methods should consider exploring the following resources. First, page seven of the Functional Behavioral Assessment online IRIS module (The IRIS Center, 2009) provides information on conducting the SIAs presented in this study. Second, the following sources provide educators with a step by step procedural checklist and a sample data sheet. Event Frequency • Event recording: Description, procedures, and example. (2003). Partial Interval • Using partial interval recording to track negative behavior. (2018, May 25). • Partial interval recording: Description, procedures, and example. (2003). Momentary Sampling • Momentary time sampling and data collection. (2019, September 18). • Momentary time sample: Description, procedures, and example. (2003). Whole Interval Sampling • Whole interval recording: Description, procedures, and example. (2003). Duration Sampling • Behavior duration: Description, procedures, and example. (2003). Latency Sampling • Latency recording: Description, procedures, and example. (2003). Finally, a practitioner focused article written by Classen and Cheatham (2015) may provide some additional information for EEs unfamiliar with SIAs. IMPLEMENTING SYSTEMATIC INFORMAL ASSESSMENT 45 Table 1. Professional Development (PD) Checklist for Improving Informal Assessment Practices The teacher education program: Evidence: 1. Incorporates informal assessment methods in assessment courses. 2. Provides readings, video examples, and opportunities to practice each informal assessment with peers. 3. Assignments require students to conduct each informal assessment method for various purposes (e.g., academic and behavioral). 4. Partners with local school districts to provide ongoing consultation and coaching. 5. Implements a program evaluation that measures teachers’ effective use of informal assessment methods post-graduation. 6. Recruit administrative feedback regarding informal assessment training needs of first year teachers. The PD provider: Evidence: 1. Incorporates PD on informal assessment methods during first two years of teaching. 2. Provides mentor teachers to serve as model informal assessment implementers. 3. Partners with the university and school district administration to provide targeted informal assessment consultation and coaching opportunities immediately following each PD session. 4. Includes a pre-post observation evaluation for each PD session on informal assessment. 5. Incorporates ongoing communication opportunities for school district administration to share informal assessment needs. 6. Develops tertiary interventions for educators struggling to incorporate informal assessment practices in their daily routines, in collaboration with the school district administration. 46 CLASSEN, KANG & CHEATHAM REFERENCES Bagnato, S. J. (2005). The authentic alternative for assessment in early intervention: An emerging evidence-based practice. Journal of Early Intervention, 28(1), 17-22. doi:10.1177/105381510502800102 Bagnato, S. J., Neisworth, J. T., & Pretti-Frontczak, K. (2010). LINKing authentic assessment and early childhood intervention: Best measures for best practice. Baltimore, MD: Paul H. Brookes Publishing Co. Banerjee, R., & Luckner, J. L. (2013). Assessment practices and training needs of early childhood professionals. Journal of Early Childhood Teacher Education, 34, 231-248. doi:10.1080/10901027.2013.816808 Behavior duration: Description, procedures, and example. (2003). Special Connections KU. Retrieved March 24, 2020 from http://www.specialconnections.ku.edu/~specconn/page/assessment/ddm/pdf/Behavior_Duration_example_r evised.pdf Catalino, T. A., & Meyer, L. E. (2015). Environment: Improving access and participation. In R. M. Santos (Ed.), DEC recommended practices: Enhancing services for young children with disabilities and their families (pp. 53-63). Los Angeles, CA: Division for Early Childhood of the Council for Exceptional Children. Classen, A., & Cheatham, G. A. (2015). Systematic monitoring of young children’s social-emotional competence and challenging behaviors. Young Exceptional Children, 18(2), 29-47. Copple, C., & Bredekamp, S. (2009). Developmentally appropriate practice in early childhood programs serving children from birth through age 8. National Association for the Education of Young Children. 1313 L Street NW Suite 500, Washington, DC 22205-4101. Division for Early Childhood (2014). DEC recommended practices in early intervention/earlychildhood special education 2014. Division for Early Childhood (DEC), (2014). DEC recommended practices in early intervention/early childhood special education 2014. Early, D. M., Maxwell, K. L., Burchinal, M., Alva, S., Bender, R. H., Bryant, D., ... & Henry, G. T. (2007). Teachers' education, classroom quality, and young children's academic skills: Results from seven studies of preschool programs. Child development, 78(2), 558-580. doi: 4139245 Event recording: Description, procedures, and example. (2003). Special Connections KU. Retrieved March 24, 2020 from http://www.specialconnections.ku.edu/~specconn/page/assessment/ddm/pdf/Event_recording_examplerevis ed.pdf Fixsen, D. L., Blase, K. A., Naoom, S. F., & Wallace, F. (2009). Core implementation components. Research on Social Work Practice, 19(5), 531-540. doi:10.1177/1049731509335549 Howes, C., James, J., & Ritchie, S. (2003). Pathways to effective teaching. Early Childhood Research Quarterly, 18, 104-120. doi:10.1016/S0885-2006(03)00008-5 Krasch, D., & Carter, D. R. (2009). Monitoring classroom behavior in early childhood: Using group observation data to make decisions. Early Childhood Education Journal, 36, 475-482. doi:10.1007%2Fs10643-009-0316-1 Latency recording: Description, procedures, and example. (2003). Special Connections KU. Retrieved March 24, 2020 from http://www.specialconnections.ku.edu/~specconn/page/assessment/ddm/pdf/latency_examplerevised.pdf Momentary time sampling and data collection (2019, September 18). How to ABA. Retrieved March 24, 2020 https://howtoaba.com/momentary-time-sampling/ Momentary time sample: Description, procedures, and example. (2003). Special Connections KU. Retrieved March 24, 2020 from http://www.specialconnections.ku.edu/~specconn/page/assessment/ddm/pdf/Momentary_Sample_exampler evised.pdf National Association for the Education of Young Children (NAEYC) & National Association of Early Childhood Specialists in State Departments of Education (NAECS/SDE) (2003). Early Childhood Curriculum, Assessment, and Program Evaluation: Building an Effective, Accountable System in Programs for Children Birth through Age 8. Neisworth, J. T., & Bagnato, S. J. (2004). The mismeasure of young children: The authentic assessment alternative. Infants & Young Children, 17, 198-212. doi:2004/07000 IMPLEMENTING SYSTEMATIC INFORMAL ASSESSMENT 47 Partial interval recording: Description, procedures, and example. (2003). Special Connections KU. Retrieved March 24, 2020 from http://www.specialconnections.ku.edu/~specconn/page/assessment/ddm/pdf/Partial_Interval_example.pdf Ramey, S. L., & Ramey, C. T. (2005). Creating and sustaining a high-quality workforce in child care, early intervention, and school readiness programs. In M. Zaslow & I. Martinez- Beck (Eds.), Critical issues in early childhood professional development (pp. 355-368). Baltimore, MD: Paul H. Brookes Publishing Co. Susman-Stillman, A., Bailey, A. E., & Webb, C. (2014). The state of early childhood assessment: Practices and professional development in Minnesota. Center for Early Education and Development, University of Minnesota. Retrieved from http://ceed.umn.edu/wp-content/uploads/2017/04/The-state-of-early-childhood- assessment-practices-and-professional-development-in-minnesota.pdf The IRIS Center. (2009). Functional Behavioral Assessment: Identifying the Reasons for Problem Behavior and Developing a Behavior Plan. Retrieved from https://iris.peabody.vanderbilt.edu/module/fba/ Using partial interval recording to track negative behavior. (2018, May 25). How to ABA. Retrieved March 24, 2020 from https://howtoaba.com/partial-interval-recording/ Waitoller, F. R., & Artiles, A. J. (2013). A decade of professional development research for inclusive education: A critical review and notes for a research program. Review of Educational Research, 83, 319-356. doi:10.3102/0034654313483905 Whole interval recording: Description, procedures, and example. (2003). Special Connections KU. Retrieved March 24, 2020 from http://www.specialconnections.ku.edu/~specconn/page/assessment/ddm/pdf/Whole_Interval_examplerevis ed.pdf Wilkins, J. L. (2008). The relationship among elementary teachers’ content knowledge, attitudes, beliefs, and practices. Journal of Mathematics Teacher Education, 11(2), 139–164. doi:10.1007/s10857-007-9068-2 Wolraich, M. L., Gurwitch, R. H., Bruder, M. B., & Knight, L. A. (2005). The role of comprehensive interdisciplinary assessments in the early intervention system. In M. J. Guralnick (Ed.), A developmental systems approach to early intervention (pp. 133–149). Baltimore, MD: Paul H. Brookes. Wortham, S. C., & Hardin, B. J. (2015). Assessment in early childhood education. Merrill/Prentice Hall.